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FOREWORD 

This  literatuare  review  has  been  performed  in  support  of  Navy  Decision  Coordinating 
Paper,  NDCP-Z0109-PN,  under  subproject  Z0109-PN.03  (Manpower  Cost  in  System 
Design)  and  was  sponsored  by  the  Deputy  Chief  of  Naval  Operations  (Manpower, 
Personnel,  and  Training,  OP-01).  The  subproject  is  directed  to  the  reduction  of  manpower 
requirements  and  associated  life  cycle  costs  of  new  hardware  systems.  Information  and 
techniques  are  being  developed  for  use  in  assisting  hardware  developers  in  assessing  the 
people-related  implications  of  their  designs  and  for  use  in  conducting  cost-effectiveness 
trade  studies  during  the  design  process. 

This  report  provides  an  annotated  bibliography  of  technical  literature  relevant  to 
identification  of  major  training,  personnel,  and  cost  variables  to  be  considered  in 
projecting  future  training  system  requirements.  The  literature  review  was  completed  late 
in  1978.  Results  are  being  used  to  develop  a  training  technology  handbook  that  will  assist 
systems  acquisition  and  training  managers  in  forecasting  training  system  course 
requirements. 

The  contracting  officer's  technical  representative  was  Mr.  John  F.  Brock. 
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INTRODUCTION 


Problem 

The  Navy  is  presently  faced  with  unprecedented  personnel  and  training  requirements. 
While  personnel  resources  continue  to  dwindle  in  number  and  quality,  rapid  increases  in 
the  sophistication  of  modern  weapon  systems  have  created  an  increasing  demand  for 
highly  trained  personnel.  Improved  methods  for  projecting  training  costs  are  needed  if  the 
Navy  is  to  implement  and  support  essential  training  programs  within  the  constraints  of 
austere  training  budgets. 

To  achieve  this  goal,  the  Navy  needs  better  cost  and  training  effectiveness 
methodologies  to  provide  both  weapons  and  training  system  managers  with  a  more  factual 
basis  for  justifying  proposed  training  programs.  Such  techniques  are  also  needed  for 
facilitating  development  of  training  programs,  estimating  the  relative  advantages  of 
alternative  training  methods  and  associated  training  devices,  and  providing  feedback  to 
the  development/design  process  for  emerging  and  future  weapon  systems.  Valid  and 
dependable  analyses  of  cost  and  training  effectiveness  are  also  needed  to  identify  basic 
and  applied  research  requirements  in  the  areas  of  training  technology,  personnel  require¬ 
ments,  and  hardware  design. 

A  major  problem  in  weapon  system  development  has  been  the  tendency  to  over¬ 
emphasize  the  performance  and  cost  of  the  hardware  subsystem,  while  underestimating 
the  importance  of  other  subsystems  such  as  personnel,  training,  and  logistics.  If  the 
systems  concept  is  to  drive  the  acquisition  process,  then  the  criterion  of  system 
effectiveness  should  be  used  to  measure  each  subsystem's  contribution  to  the  overall 
system  performance.  Only  in  this  way  can  correct  and  timely  trade-off  analyses  be 
conducted  at  the  proper  points  in  the  weapons  system  acquisition  process  (WSAP)  to 
ensure  maximum  system  success. 


The  current  difficulty  in  projecting  training  costs  and  requirements  can,  in  part,  be 
attributed  to  the  complexity  of  the  weapons  system  acquisition  process.  One  measure  of 
this  complexity  is  the  number  and  variety  of  official  Navy  and  DoD  directives  pertaining 
to  the  process.  The  1975  issue  of  the  Department  of  the  Navy  RDT&E  Management  Guide 
lists  more  than  200  directives  as  references.  The  training  system  acquisition  process  is 
similarly  complex. 

Until  recently,  training  system  requirements  were  rarely  addressed  in  detail  until  late 
in  the  development  of  operational  systems.  System  development  was  viewed  as  a 
sequence  of  events  in  which  the  determination  of  training  requirements  was  a  separate 
and  distinct  activity.  Furthermore,  training  programs  (and  training  device  designs)  were 
typically  determined  solely  on  the  characteristics  of  operational  systems  rather  than 
being  driven  by  training  objectives.  This  disjointed  process  failed  to  recognize  and  realize 
the  mutual  benefits  inherent  in  an  integrated  approach  to  weapon  and  training  system 
development. 

This  situation  has  received  attention  in  the  last  few  years.  In  a  memorandum  for 
secretaries  of  the  military  departments  dated  17  August  1978,  the  Assistant  Secretary  of 
Defense  called  for  greater  emphasis  in  early  forecasting  of  manpower  needs  for  new 
weapon  systems.  Manpower  needs  were  defined  as  "personnel  assets,  spaces,  policies, 
planning,  training  and  utilization  (including  support  policies  and  equipment,  capital/labor 
trade-offs,  technical  data,  job  classifications,  and  organizations)." 

Objective 


The  overall  objective  of  this  effort  is  to  develop  a  handbook  for  systems  acquisition 
and  training  managers  that  will  provide  them  with  the  tools  to  make  reasonable  estimates 
of  the  cost  and  composition  of  training  required  for  new  hardware  systems  while  in  the 


The  objective  of  the  effort  described  herein  was  to  review  the  state  of  the  art  in 
instructional  systems  design  (ISD),  training  management  systems,  and  training  system  cost 
effeciveness. 

APPROACH 

Reviews  were  conducted  in: 

1.  The  literature  on  weapon  system  and  training  subsystem  development  to  identify 
potential  sources  of  difficulty  in  system  planning  and  development. 

2.  The  literature  on  training  subsystems,  with  particular  attention  directed  toward 
current  developments  in  task/skill  analysis,  media  selection  methods,  instructional 
delivery  systems  (including  individual  differences,  adaptive  training,  and  team  training), 
and  training  program  assessment. 

3.  The  literature  on  training  cost  estimation. 

RESULTS 

The  annotated  literature  review  produced  is  provided  in  Appendix  A.  Each  reference 
citation  is  accompanied  by  a  description  of  the  reference  contents.  References  selected 
for  annotation  provide  a  comprehensive  sample  of  the  range  of  current  literature. 
Additonal  references,  including  some  older,  less  critical  documents,  are  provided  in 
Appendix  B. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  data  presented  here  must  be  refined  and  further  evaluated  to  identify  relation¬ 
ships  and  interactions  among  categories  of  data  (e.g.,  the  relationship  between  particular 
instructional  strategies  and  the  cost  of  training  programs).  Furthermore,  the  appropriate¬ 
ness  of  particular  instructional  strategies  should  be  assessed  in  relation  to  such  factors  as 
student  population  characteristics,  the  frequency  and  duration  of  the  instruction,  require¬ 
ments  for  individualized  and  team  training,  the  location  of  the  instruction  (03T,  school, 
etc.),  and  the  purpose  of  instruction  (initial  training,  refresher  training,  transfer  of 
training  to  new  equipment,  etc.). 


APPENDIX  A 

ANNOTATED  LITERATURE  REVIEW 


System  Concepts . A- 1 

Training  Subsystem . A- 11 

Task  and  Skill  Analysis . A- 11 

Media  Selection  Methods . A-23 

Training  Program  Assessment . A-31 

Instructional  Delivery  Systems . A-35 

Training  Cost  Estimation . A-51 


ANNOTATED  LITERATURE  REVIEW 


System  Concepts 

Air  Force  Systems  Command.  Human  factors  engineering:  Personnel  subsystem  (2nd  Ed., 
AFSC  design  handbook  1-3).  Washington,  DC:  Andrews  Air  Force  Base,  January  1972. 

This  handbook  documents  Air  Force  technical  knowledge  on  the  considerations  and 
characteristics  of  the  personnel  subsystem  that  have  bearing  on  the  design,  operation  and 
maintenance  of  Air  Force  equipment  and  systems.  The  areas  covered  include:  human 
engineering,  which  is  the  information  on  the  physical  capabilities  and  limitations  of  men 
used  in  the  design  of  systems  and  equipment  to  ensure  safe,  reliable,  and  efficient 
personnel  performance;  the  biomedical  and  physiological  criteria  used  in  system  design  to 
promote  health,  safety  and  physical  protection  of  personnel;  the  personnel  and  manning 
factors  that  bear  on  the  types  and  numbers  of  personnel  required  for  system  operation  and 
maintenance;  the  training  and  training  equipment  considerations  (not  complete);  the  job 
performance  aids  that  must  support  task  performance  within  the  context  of  technician 
capabilities  and  equipment  characteristics;  and  the  development  of  measures  of  perform¬ 
ance  and  proficiency  (not  complete). 

Betaque,  N.,  Kennelly,  D.,  &  Nauta,  F.  Navy  manpower  planning  for  new  weapon  systems 
(Tech.  Rep.  WN  Ml 80 1-1).  Washington,  DC:  Logistics  Management  Institute,  August 
1978. 

This  working  note,  which  describes  how  the  Navy  establishes  manpower  needs  for  new 
weapon  systems,  begins  by  examining  how  the  Navy  creates,  funds,  and  fills  billets 
(manpower  authorization).  Billets  for  ships  are  derived  from  requirements  contained  in 
ship  manning  documents  (SMDs).  These  documents  are  drafted  late  in  the  acquisition 
process  and  finalized  a  year  after  deployment  of  a  system.  The  SMD  describes  minimal 
billets  to  meet  required  operational  capabilities  (ROC)  based  on  the  mission  phase 
demanding  the  most  manpower.  Final  billets,  however,  often  are  less  than  this  based  on 
the  Navy  end  strength.  Deputy  Chiefs  of  Naval  Operations  (DCNOs)  are  mission  or 
warfare  sponsors  that  are  responsible  for  programming  billets.  DCNO  (Surface  Warfare) 
has  this  responsibility  for  most  surface  ships.  There  is  no  formal  process  for  deriving 
authorizations  from  requirements.  DCNO  (Surface  Warfare)  provides  input  to  the  Navy 
Program  Objective  Memorandum  (POM)  in  the  form  of  sponsor  program  proposals  (SPPs) 
once  a  year.  The  SPP  for  training  is  prepared  by  the  Chief,  Naval  Education  and  Training 
(CNET)  with  assistance  of  a  Training  Resources  Panel  (TRP). 

Manpower  planning  as  related  to  the  weapon  system  acquisition  process  is  considered 
next.  Key  documents  discussed  include  the  operational  requirement,  development 
proposal,  Navy  decision  coordinating  paper,  and  the  integrated  logistical  support  plan. 

Separate  sections  are  devoted  to  the  determination  of  manpower  requirements  for 
surface  ships,  aircraft,  and  strategic  weapon  systems.  Whereas  most  conceptual  develop¬ 
ment  of  ships  is  done  in-house  by  the  Naval  Ship  Engineering  Center  (NAVSEC), 
development  of  aircraft  is  done  by  private  industry.  For  surface  ships,  NAVSEC  estimates 
of  manpwer  requirements  are  based  on  the  Manpower  Determination  Model  (MDM).  For 
aircraft,  manning  is  estimated  by  the  Naval  Aviation  Logistics  Center  (NALC)  based  on 
Navy  data  on  similar  aircraft,  contractors'  estimates,  test  results,  and  the  experienced 
analyst's  judgment.  NAVSEC  and  NALC  methods  of  estimating  manpower  requirements 
are  described  in  detail  in  the  appendices. 

The  relatively  large  portion  of  the  report  devoted  to  manpower  planning  for  strategic 
weapon  systems  reflects  the  authors'  interest  in  the  unique  organization  of  the  Strategic 
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Systems  Project  Office  (SSPO).  Emphasis  is  on  training  system  development,  because  the 
training  system  project  manager  has  authority  equal  to  that  of  the  weapon  system 
(hardware)  project  manager.  This  ensures  that  manpower  and  training  considerations  are 
weighted  heavily  during  all  phases  of  weapon  system  development. 

The  authors  see  two  major  weaknesses  in  the  Navy  acquisition  review  process. 
Except  for  strategic  systems,  weapon  system  project  managers  have  not  been  required  to 
integrate  manpower  and  training  considerations  into  weapon  system  development.  As  a 
result,  little  attention  has  been  given  to  these  concerns.  The  second  problem  in  the  past 
is  that  the  DCNO  (Manpower)  was  not  involved  in  the  acquisition  review  process.  The 
DCNO  (Logistics)  was  responsible  for  manpower  considerations,  but  focused  attention  on 
more  traditional  logistics  concerns.  Recently,  a  HARDMAN  office  was  established  to 
evaluate  manpower  considerations,  which  should  help  correct  the  second  problem. 

Current  methods  of  predicting  manpower  needs  (e.g.,  MDM,  OPNAV  10P-23)  should 
be  refined  to  encompass  needs  for  each  readiness  condition,  rather  than  computing 
average  needs.  Also,  the  authors  suggest  that  the  guidelines  in  OPNAV  10P-23  are  at 
times  confusing,  and  some  of  the  measures  are  insensitive  and  do  not  provide  for  manning 
of  manual  backups. 

Demaree,  R.  G.,  Marks,  M.  R.,  Smith,  W.  L.,  <5c  Snyder,  M.  T.  Development  of  qualitative 
and  quantitative  personnel  requirements  information  (Tech.  Rep.  AMRL-TDR-62-4). 
W right-Pa tterson  Air  Force  Base,  OH:  6570  Personnel  Research  Laboratory,  Aerospace 
Medical  Division,  December  1962. 

This  report  describes  the  technical  aspects  of  qualitative  and  quantitative  personnel 
requirements  information  (QQPRI)  development.  The  relationship  of  QQPRI  development 
to  system  development  is  defined,  and  general  constraints  impacting  on  QQPRI  develop¬ 
ment  are  identified.  However,  the  major  component  of  the  report  is  a  description  of  the 
specific  analysis  occurring  during  QQPRI  development.  QQPRI  is  a  formal  report  required 
for  personnel  subsystem  development  that  contains  manning  estimates,  describes  duty 
position,  and  defines  the  maintenance  and  control  of  the  operational  equipment.  A  key 
role  of  the  QQPRI  is  to  assure  compatibility  of  hardware  capabilities  and  procedures  with 
the  characteristics  of  the  personnel  assigned  to  the  system.  QQPRI  reports  can  also  be 
used  to  project  realistic  long  range  personnel  requirements.  Early  emphasis  is  based  in 
large  part  upon  the  logical  progression  of  time-based  mission  profiles,  function  defini¬ 
tions,  man-machine  allocation  of  functions,  and  task  performance  requirements  for 
operation  and  maintenance  of  the  weapon  systems.  Subsequent  activities  focus  on 
detailed  analysis  of  individual  task  requirements.  It  must  then  be  determined  whether 
these  job  duties  fit  within  an  existing  job  specialty  or  a  new  specialty  is  required. 

Department  of  the  Air  Force.  Instructional  System  Development  (AF  Manual  50-2). 
Washington,  DC:  July  1975. 

This  manual  presents  the  concepts  and  the  Air  Force  policy  for  instructional  systems 
development  (ISD).  (Detailed  procedures  for  implementation  of  ISD  concepts  are  given  in 
AFM  50-58.)  ISD  is  a  systematic  process  that  ensures  application  of  instructional 
technology  to  cause  planning  and  development.  Air  Force  policy  is  that  ISD  will  be  used 
for  all  new  instructional  systems  and,  where  feasible,  for  modification  of  existing  ones.  A 
five-step  procedure  model  of  ISD  is  presented:  Analyze  system  requirements,  define 
education/training  requirements,  develop  objectives  and  tests,  plan/develop/ validate  in¬ 
struction,  conduct  and  evaluate  instruction.  These  steps  are  similar,  but  not  identical,  to 
the  five  steps  in  the  interservice  procedures  ISD  model.  Some  features  particularly  the 
media  selection  of  the  Air  Force  ISD  model  differ  from  the  interservice  procedures.  In 


the  Air  Force  procedure,  a  type-of-learning  by  instructional-media  matrix  is  presented. 
Appropriateness  of  media  for  each  type  of  learning  is  assessed  on  three  scales: 
presentation,  practice,  and  feedback.  The  Interservice  procedure  assigns  appropriateness 
of  media  on  only  a  single  scale.  Whereas  the  Air  Force  procedure  considers  only  6  types 
of  learning,  the  Interservice  procedure  considers  11.  The  Air  Force  manual  contains  a 
separate  chapter  on  the  application  of  ISD  to  knowledge  and  attitudes.  Also,  there  is  a 
chapter  on  Air  Force  technical  training  that  describes  the  Air  Force  personnel  classifi¬ 
cation  system. 

Department  of  the  Navy.  Department  of  the  Navy  RDT&E  management  guide  (NAVSO  P- 
2457,  Rev.  1-75).  Washington,  DC:  U.S.  Government  Printing  Office,  1975. 

This  guide  was  developed  to  provide  both  experienced  and  inexperienced  R&D 
personnel  with  a  comprehensive  overview  of  the  RDT&E  process,  directives,  and  termin¬ 
ology. 

Gerry,  R.  Learning  systems:  Requirements  and  technology.  The  Second  Symposium  on 
Cost  Effective  Learning  Through  the  Application  of  Computer  Technology,  Washington, 
DC,  February  1973,  143-153. 

The  author  cites  many  inadequacies  in  the  general  individualized  instructional  system 
model.  Specifications  of  requirements  is  a  major  deficiency.  The  author  feels  that  if 
efforts  are  made  to  attend  to  this  need,  more  precise  counseling  between  people,  learning, 
and  the  working  world  will  be  made  possible  with  additional  benefits  in  terms  of  personal 
motivation,  interest,  and  confidence  in  work  assignment  and  productivity. 

Hanson,  V.  L.,  &  Purifoy,  G.  R.,  Jr.  TSM  guide  to  training  development  and  acquisition 
for  major  systems  (Tech.  Rep.  78-A7T!  Alexandria:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences,  March  1978. 

This  guide  provides  TRADOC  System  Managers  (TSM)  with  an  integrated  view  of 
weapon  system  development.  Training  subsystem  development  is  highlighted  as  the  major 
focus  to  ensure  that  training  considerations  occur  in  a  timely  manner  during  development 
of  the  weapon  system.  Training  Development  and  the  life  cycle  system  management 
model  (LCSMM)  are  presented  separately,  and  are  followed  by  a  discussion  of  integrating 
training  subsystem  acquisition  within  the  LCSMM.  Training  related  events  are  detailed 
for  each  major  phase  of  weapon  system  development. 

Hirshfeld,  S.  F.,  &  Kochevar,  J.  W.  Training  device  requirements  documents  guide  (draft): 
A  procedures  handbook  for  directorate  of  training  developments  project  officers  for 
devices  (DTDPQDTI  Minneapolis:  Honeywell  Systems  and  Research  Center,  October 
1978. 


The  Training  Device  Requirements  Documents  Guide  was  developed  to  aid  Training 
and  Doctrine  Command  (TRADOC)  school  personnel  in  the  development  of  training 
devices.  The  procedures  handbook  specifies  how  to  (1)  define  training  requirements  based 
on  data  from  front-end  analysis  and  (2)  perform  media  analyses.  Both  of  these  activities 
are  required  prior  to  initiating  the  conceptual  development  of  a  training  device.  The 
handbook  presents  procedures  and  decision  rules  for  determining  when  a  training  device  is 
required,  and  when  alternative  media  are  more  appropriate.  When  a  training  device  is 
required,  the  procedures  handbook  specifies  how  ini  tied  training  device  concepts  are 
formalized,  validated,  and  specified  in  requirements  documents.  Detailed  examples  are 
included  to  illustrate  each  type  of  analysis  and  requirements  document. 


Johnson,  S.  L.,  Knight,  J.  R.,  &  Sugarman,  R.  C.  Simulation  technology  assessment  report 
(B-l  Technical  Memorandum  SAT-3).  Buffalo,  NY:  Calspan  Corporation,  July  1975. 


Sugarman,  R.  C.  Defining  the  system  in  the  systems  approach  to  training.  Conference  on 
Issues  in  Air  Traffic  Control  Simulation,  Training,  and  Technology,  Second  International 
Learning  Technology  Congress  and  Exposition,  February  1978. 

Laughery,  K.  R.,  &  Sugarman,  R.  C.  A  systems  approach  to  the  design  of  training  devices: 
A  model  and  its  application.  Proceedings  of  the  Human  Factors  Society  22nd  Annual 
Meeting,  Detroit,  October  1978. 

In  a  report  delivered  to  the  Air  Force  as  part  of  the  B-l  instructional  system 
development  by  Johnson,  Knight,  and  Sugarman  (1975)  and  later  amplified  by  Sugarman 
(1978),  the  instructional  system  design  process  was  conceived  as  a  flow  of  information. 
The  flow  of  information  necessary  to  design  an  efficient  training  system  must  highlight 
numerous  interactions  among  facts,  assertions,  assumptions,  and  constraints.  Although 
the  proposed  model  presents  a  series  of  chronological  steps,  it  explicitly  recognizes  the 
iterative  nature  of  the  instructional  system  design  process.  The  necessity  for  iteration  in 
this  process  is  due  to  the  large  interactions  between  methods/media  specification, 
instructional  block  and  course  allocation,  and  system  requirements  and  constraints. 
Rather  than  an  algorithm  for  the  design  of  training  systems,  this  report  identifies 
framework  concepts  to  be  used  during  the  design  of  training  systems.  This  model  clearly 
recognized  the  strong  interdependencies  that  exist  between  course  design  (e.g.,  syllabus) 
and  resource  requirements  and  constraints  (e.g.,  training  aids,  dollar  limitations). 

Later,  this  systematic  approach  to  training  system  design  was  focused  to  deal 
specifically  with  the  design  of  training  devices  by  Laughery  and  Sugarman  (1978).  The 
model  presented  in  this  article  recognizes  that  several  types  of  data  generated  during  the 
instructional  system  development  process,  yield  valuable  information  about  training 
device  requirements.  First,  a  task  analysis  is  normally  conducted  that  permits  the 
training  analyst  to  examine  relatively  small  units  of  the  human's  responsibilities  in  overall 
system  operations.  This  enables  the  training  analyst  to  avoid  unnecessary  training  while 
still  ensuring  that  all  essential  tasks  are  properly  trained.  On  the  basis  of  the  task 
analysis  coupled  with  an  analysis  of  the  expected  incoming  skills  and  knowledges  of 
students,  a  set  of  training  objectives  is  generated.  The  methodology  for  designing  a  set  of 
training  devices  employs  this  training  objective  data  base  for  subsequent  analyses.  Each 
training  obj  ~+'',e  is  placed  into  one  of  five  categories  indicating  the  basic  type  of 
environment  *xl  for  the  objective  to  be  properly  taught;  (1)  cognitive  (e.g., 

classroom),  (2)  *  ;arization  (to  permit  spatial  relationships  to  be  determined),  (3) 
procedural  (to  pern..  npsychomotor  procedures  using  controls  and  displays  to  be  taught 
and  practiced),  (4)  tu.!y  interactive  (to  permit  psychomotor,  as  well  as  procedural 
components  of  the  task  to  be  practiced),  and  (5)  the  operational  environment  itself  (where 
the  state  of  simulation  technology  cannot  provide  the  appropriate  environment  artificial¬ 
ly).  Familiarization,  procedural,  and  fully  interactive  training  objectives  may  lend 
themselves  to  training  in  a  training  device  while  cognitive  objectives  require  no  devices 
and  operational  equipment  requires  no  specification  based  on  training  requirements. 

For  each  of  the  familiarization,  procedural,  and  fully  interactive  training  objectives 
an  analysis  is  made  to  determine  the  specific  environmental  characteristics  necessary  to 
train  that  objective.  This  is  done  by  using  judgments  regarding  the  controls  and  displays 
involved,  the  extent  to  which  they  must  be  operable,  and  what  other  identifiable 
environmental  cues  are  relevant  to  facilitate  learning  of  that  objective  (e.g.,  visual, 
auditory,  vestibular). 


paper  concludes  that  educators  should  spend  more  time  developing  individual  memory  and 
problem-solving  skills. 

Kurke,  M.  I.  Operational  sequence  diagrams  in  system  design.  Human  Factors,  1961,  3, 
66-73. 

The  operational  sequence  diagram  approach  presents  information/decision/action 
sequences  required  of  a  system,  or  its  constituent  component  systems,  to  accomplish 
prescribed  mission  objectives.  It  has  been  used  to  establish  sequence  of  operations 
requirements  between  subsystem  interfaces  at  various  levels  of  system  analysis,  to 
identify  problem  situations  traceable  to  inadequate  hardware,  to  establish  requirements 
for  training  programs  and  associated  devices,  and  to  provide  information  relevant  to 
human  factors  considerations  of  workspace  layout  and  control  and  display  design.  It  has 
been  applied  to  programs  dealing  with  instrumentation  radar,  fire  control  systems,  and 
other  types  of  weapon  systems. 

Lumsdaine,  A.  A.  Design  of  training  aids  and  devices.  In  3.  D.  Folley,  Jr.  (Ed.),  Human 
factors  methods  for  system  design  (Tech.  Rep.  C90-60-FR-225).  Pittsburgh:  The 
American  Institute  of  Research  for  Office  of  Naval  Research,  1960,  217-290. 

The  steps  to  be  followed  in  conducting  analyses  for  training  devices  are  (1)  determine 
the  human  performance  requirements  for  a  job;  (2)  derive  training  requirements  in  terms 
of  skills  and  knowledge  to  be  acquired;  (3)  consider  logisitic  requirements  and  total 
training  plans;  and  (4)  use  a  mockup  of  a  device  to  obtain  an  informal  evaluation  of  the 
procedures  developed.  The  procedures  are  revised  based  on  the  tryout  results. 

Marquardt,  L.  D.,  &  McCormick,  E.  J.  Attribute  ratings  and  profiles  of  the  job  elements 
of  the  Position  Analysis  Questionnaire  (PAQ)  (Tech.  Rep.  1).  Lafayette,  IN:  Purdue 
University,  Occupational  Research  Center,  June  1972. 

The  purpose  of  this  study  was  to  obtain  estimates  of  reliability  of  the  human 
attribute  requirements  of  form  B  of  the  PAQ.  Eighty  raters  rated  the  PAQ  job  elements 
using  a  new  list  of  76  attributes.  There  is  evidence  that  situational  attributes  have  a 
lower  reliability  than  aptitude  attributes. 

Marquardt,  L.  D.,  &  McCormick,  E.  J.  Component  analysis  of  the  attribute  data  based  on 
the  Position  Analysis  Questionaire  (PAQ)  (Tech.  Rep.  2).  Lafayette,  IN:  Purdue 
University,  Occupational  Research  Center,  1973. 

This  paper  extends  the  work  on  the  attribute  profiles.  Sixty-seven  human  attributes 
judged  as  relevant  to  work  behavior  were  judged  in  terms  of  their  relevance  to  the 
elements  of  the  PAQ.  The  underlying  structure  of  human  attributes  (abilities  and 
situational  requirements)  is  analyzed  in  terms  of  co-existence  on  the  job,  while,  the 
underlying  structure  of  job  elements  is  analyzed  in  terms  of  relevance  to  human 
attributes.  There  seems  to  be  no  problem  in  using  this  approach. 

Marquardt,  L.  D.,  <5c  McCormick,  E.  J.  The  job  dimensions  underlying  the  job  elements  of 
the  Position  Analysis  Questionnaire  (PAQ)  (Tech  Rep.  4).  Lafayette,  IN:  Purdue 
University,  Occupational  Research  Center,  June  1974. 

This  study  is  concerned  with  the  identification  of  the  job  dimensions  underlying  the 
job  elements  of  the  PAQ  (Form  B),  which  is  a  structured  job  analysis  instrument  consisting 
of  187  worker-oriented  job  elements  divided  into  six  dimensions.  The  dimensions 
identified  are  based  on  a  representative  sample  of  jobs. 


personality  variables  and  achievement  measures,  but  not  with  job  behavior.  The 
procedures  used  to  support  the  existence  of  the  120  abilities  are  too  flexible  to  be 
conclusive. 

Gustafson,  H.  W.,  Honsberger,  W.  D.,  Sc  Michelson,  S.  Determination  of  task  analysis 
content.  In  Uses  of  task  analysis  in  deriving  training  and  training  equipment  requirements 
(Tech.  Rep.  60-593).  Wright-Patterson  Air  Force  Base,  OH:  Wright  Air  Development 
Center,  December  1960. 

These  authors  believe  that  the  content  of  task  analysis  cannot  be  widely  generalized 
from  problem  to  problem.  They  argue  that  although  a  universally  applicable  task  analysis 
format  can  never  be  attained,  a  generalizable  task  analysis  procedure  can  be  derived. 
They  consider  the  decisions  that  have  to  be  made  about  the  design  of  training  and  training 
equipment  for  a  specific  system,  and  list  questions  about  task  attributes  that  have  to  be 
addressed  prior  to  decision  making.  In  practice  the  method  has  had  only  limited  success, 
but  justifies  further  scrutiny. 

Hammel,  T.,  Sroka,  F.  P.,  Sc  Allen,  F.  L.  Study  of  training  device  needs  for  meeting  basic 
officer  tactics  training  requirements.  Groton,  CT:  General  Dynamics,  Electric  Boat 
Division,  March  1971. 

A  diagramming  method  called  link  analysis  is  applied  to  portray  communication  links 
between  team  members  during  the  conduct  of  mission  activities.  These  diagrams  convey 
either  the  strength  (importance)  or  frequency  of  the  communications. 

Hughes,  3.  A.,  Sc  Hymes,  3.  P.  A  study  of  the  effectiveness,  feasibility  and  resource 
*  requirements  of  instructional  systems  development:  EA-6B  readiness  training  (Tech. 
Rep.  75-C-0100-1).  San  Diego:  Courseware  Incorporated,  3anuary  1977. 

Courseware,  Inc.  has  developed  a  process  of  analyzing  the  human  functions  performed 
in  a  system  into  constituent  major  tasks.  A  listing  of  the  tasks  performed  by  each 
member  of  a  team  is  developed.  These  listings  serve  as  the  foundation  for  subsequent 
system  analysis  efforts.  Relationships  among  tasks  are  represented  in  a  standard  format 
which  is  sufficiently  flexible  to  allow  changes  that  require  minimal  reorganization.  The 
task  lists  are  derived  by  first  having  subject  matter  experts  describe  a  step-by-step 
account  of  a  mission.  Instructional  psychologists  and  technologists  work  with  the  expert 
in  more  formally  defining  the  tasks  performed  during  conduct  of  the  mission. 

Hunt,  E.  We  know  who  knows,  but  why?  In  R.  C.  Anderson,  R.  3.  Spiro,  <5c  W.  E.  Montague 
(Eds.),  Schooling  and  the  acquisition  of  knowledge;  Conference  proceedings  (NPRDC 
Tech.  Rep.  78-6).  San  Diego:  Navy  Personnel  Research  and  Development  Center, 
December  1977. 

Some  of  the  implications  of  individual  differences  in  information  processing  are 
considered.  It  is  possible  to  predict  the  ability  of  an  individual  to  acquire  new  knowledge. 
Since  knowledge  builds  on  previously  acquired  knowledge,  small  differences  in  information 
processing  can  multiply  into  large  differences  in  acquired  knowledge.  A  model  of 
linguistic  information  processing--the  Carpenter  and  3ust  constituent  comparison 
model--is  presented,  and  research  findings  on  individual  differences  in  reaction  time  are 
used  to  explain  learning  differences  in  terms  of  the  model.  Three  areas  requiring  more 
research  are  presented:  (1)  How  do  information  processing  measures  relate  to  each  other? 
(2)  What  strategies  are  used  by  skilled  problem  solvers  in  selecting  pertinent  information 
and  generating  alternate  representations?  (3)  How  do  the  relative  contributions  of 
information  processing  and  knowledge  change  as  the  tasks  and  the  subjects  change?  The 


4.  Instructions  on  what  information  to  obtain,  how  to  get  it,  and  where  and  how  to 
record  it. 

5.  Examples  of  completed  portions  of  a  task  analysis. 

This  report  also  presents  a  theory  showing  the  relationship  between  the  required  task 
data  and  the  training  requirements  determined  from  that  task  data. 

French,  J.  W.  Toward  the  establishment  of  non-cognitive  factors  through  literature 
search  and  interpretation  (Tech.  Rep.  Q.  Princeton,  N3:  Educational  Testing  Service, 
July  1973. 

This  report  describes  one  study  in  a  series  involving  the  conceptualization  of  ability 
and  temperament  factors  and  the  development  of  measures  of  these  factors.  This 
particular  study  was  a  literature  review  of  temperament  factors:  Twenty-eight  factors 
selected  from  the  literature  were  not  empirically  tested. 

Gibbons,  A.  S.,  &  Hughes,  J.  A.  A  method  of  deriving  hierarchies  of  instructional 
objectives.  Proceedings  of  the  Human  Factors  Society  22nd  Annual  Meeting,  Detroit, 
October  1978,  256-259. 

The  establishment  of  objectives  hierarchies  by  analysts  has  been  very  subjective  and 
nonuniform.  The  procedures  presented  herein  are  intended  to  enable  the  inexperienced 
analyst  to  develop  logically  consistent  objectives  hierarchies.  These  hierarchies  are 
constructed  by  first  obtaining  task  lists  then  subjecting  these  tasks  to  finer  analysis,  and, 
finally  by  using  a  set  of  rules  to  transform  these  subtasks  into  instructional  objectives  in 
an  orderly  fashion.  Tasks  are  listed  at  levels  of  detail  sufficient  for  performance 
standards  and  conditions  to  be  specified.  Subtasks  are  then  classified  as  either 
classification  and  rule-using  or  motor  control.  Verbal  support  requirements  for  each 
subtask  are  identified.  In  addition,  minor  subtasks  are  grouped  in  a  way  that  provides 
meaningful  instruction  for  the  group,  when  separate  instruction  for  each  would  not  be 
appropriate.  Those  minor  subtasks  that  cannot  be  grouped  are  dropped  from  the  list  of 
tasks  to  be  trained.  Tasks  that  are  too  expensive  or  too  dangerous  to  train  are  eliminated. 
Finally,  for  those  subtasks  that  might  require  nonstandard  procedures  in  special  situations, 
an  analysis  of  the  knowledge  base  is  performed  to  provide  performance  guidelines.  The 
analyst  performs  each  of  the  nine  steps  for  each  task  in  the  task  list.  Through  this 
procedure,  patterns  of  complex  behavior  and  their  enabling  objectives  are  identified. 
These  patterns  identify  prerequisite  instruction  to  task  performance.  These  methods  are 
intended  to  guide  the  thoughtful  analyst  and  ensure  that  some  minimal  considerations  are 
made. 

Goodyear  Aerospace  Corporation.  Study  of  submarine  casualty  control  training  (Tech. 
Rep.  1813-1).  Port  Washington,  NY:  Naval  Training  Device  Center,  March  1966. 

This  approach  is  used  to  analyze  casualty  data.  It  attends  to  crew  responsibilities, 
conditions,  and  performance,  and  provides  an  analysis  of  casualty  sequences  for  a 
particular  system.  Information  flow  diagrams  are  used  to  represent  a  linear  analysis  of 
information  flow  relative  to  a  specific  system.  The  information  flow  does  not  key  on 
particular  individuals  processing  the  information. 

Guilford,  J.  P.,  ic  Hoepfner,  R.  The  analysis  of  intelligence.  New  York:  McGraw-Hill, 
1971. 


Guilford’s  research  on  intellectual  abilities  is  summarized  and  his  model  of  the 
structure  of  abstract  cognitive  abilities  is  presented.  The  abilities  are  correlated  with 


Fleishman,  E.  A.,  Kinkade,  R.  G.,  <5c  Chambers,  A.  N.  Development  of  a  taxonomy  of 
human  performance:  A  review  of  the  first  year's  progress  (Tech.  Rep,  l).  Washington, 
DC:  American  Institutes  for  Research,  November  1968. 

This  report  reviews  the  first  year's  progress  of  a  5-year  project  to  develop  and  verify 
the  utility  of  a  taxonomic  system  of  human  task  performance.  It  is  organized  into  four 
areas:  the  review  of  previous  taxonomic  efforts,  the  development  of  an  integrative 
model,  the  proposal  of  a  provisional  classification  scheme,  and  the  definition  of 
requirements  for  a  human  performance  data  base.  There  are  no  empirical  findings  and,  as 
the  report  covers  the  first  year  of  a  5-year  project,  the  utility  is  only  projected. 

Fleishman,  E.  A.,  <5c  Stephenson,  R.  W.  Development  of  a  taxonomy  of  human 
performance:  A  review  of  the  third  year's  progress.  Washington,  DC:  American 
Institutes  for  Research,  1970. 

This  report  is  of  the  third  and  final  year  of  what  was  originally  a  5-year  project. 
Only  brief  examples  are  given  of  the  constructed  task  assessment  scale  (describing  tasks), 
the  task  characteristics  rating  scales,  information  theory  categories,  definitions,  and  data 
base  categories.  Empirical  findings  are  not  reported. 

Fleishman,  E.  A.,  Teichner,  W.  H.,  <3c  Stephenson,  R.  W.  Development  of  a  taxonomy  of 
human  performance:  A  review  of  the  second  year's  progress.  Washington,  DC: 
American  Institutes  for  Research,  1969. 

Three  provisional  approaches  (each  relevant  to  one  user  type)  to  performance 
taxonomy  are  developed:  ability  requirements,  task  characteristics,  and  systems  language 
approaches.  Classifications  are  made  in  terms  of  37  human  abilities,  24  task  characteris¬ 
tics  and  7  types  of  information  processing  constraints.  Taxonomies  are  not  validated  or 
utilized. 

F.M.C.  Corporation.  MICV/TBAT-II,  human  factors  engineering  progress  report,  sequence 
AD  17,  DI-H-1314  (Tech.  Rep.  DAAK  30-77-C-0002).  San  Jose.  CA:  May  1977. 

F.M.C.  Corporation  devised  a  task  sequence  analysis  approach  in  which  the  major 
functions  and  tasks  performed  by  system  personnel  are  identified  and,  then,  sequentially 
ordered.  A  system  mockup  is  developed  and  used  by  subjects  in  a  series  of  trials.  The 
subjects  perform  a  series  of  tasks  described  in  a  written  script.  Human  factors  personnel 
observe  these  trials  and  record  time,  duration,  and  position  of  subjects  during  task 
performance.  These  data  are  then  used  during  further  refinements  in  the  training  system 
development  process. 

Folley,  J.  D.,  Jr.  Development  of  an  improved  method  of  task  analysis  and  beginnings  of  a 
theory  of  training  (Tech.  Rep.  1218-1).  Orlando:  Navy  Training  Device  Center,  June 
1964. 

The  objective  of  this  study  was  to  devise  an  effective  method  of  task  analysis  that 
would  fit  in  with  a  larger  program  of  training  device  development.  The  guidelines  for  task 
analysis  that  were  defined  include: 


1.  Background  materials  thought  to  be  useful  to  the  task  analyst. 

2.  An  overview  of  the  process  of  task  analysis. 

3.  Samples  of  forms  to  be  used  in  task  analysis. 


development  process  because  they  relate  to  the  content  of  a  job,  associated  tasks  and 
duties,  mission  requirements,  equipment  and  supporting  materials,  allocation  of  man- 
machine  functions,  operator  capabilities,  and  other  general  personnel  policies,  concepts, 
and  manpower  forecasts.  Therefore,  the  personnel  information  becomes  a  characteriza¬ 
tion  of  a  particular  job  and  its  associated  responsibilities. 

Demaree,  R.  G.,  Norman,  D.  A.,  <Jc  Matheny,  W.  G.  An  experimental  program  for  relating 
transfer  of  training  to  pilot  performance  and  degree  of  simulation.  (Tech.  Rep.  7619-1). 
Fort  Worth:  Life  Sciences,  1965.  (AD-471  806) 

Training  equipment  requirements  data  are  gathered  to  generate  functional  require¬ 
ments  for  a  training  situation.  Operational  tasks  are  analyzed  into  training  function 
statements  of  what  the  trainee  needs  to  know.  Each  training  function  is  then  divided  into 
discrete  tasks  and  fidelity  requirements  are  derived.  Finally,  performance  standards  are 
established  for  each  task.  Considerations  of  trainee  characteristics,  training  context, 
equipment  requirements,  and  instructor  requirements  are  all  taken  into  account. 

Department  of  the  Air  Force.  Handbook  for  designers  of  instructional  systems,  Volume  II 
(Tech.  Rep.  50-58).  Washington,  DC:  Author,  July  1973. 

The  handbook  consists  of  five  volumes  and  addresses  the  development  of  instructional 
systems.  The  volumes  include:  introduction  task  analysis;  objectives  and  tests;  planning; 
developing  and  validating  instruction;  and  evaluation.  In  Volume  II,  task  decision 
worksheets  are  provided  to  record  task  information.  The  statement  includes  identifying 
information,  training  factors,  and  task  activity  factors.  The  format  can  be  an  outline,  a 
flow  chart,  a  table,  or  a  time  line.  This  document  is  a  forerunner  to  the  ISD  model. 

Ekstrom,  R.  B.,  French,  J.  W.,  &  Harman,  H.  H.  Problems  of  replication  of  seven 
divergent  production  factors  (Tech.  Rep.  5).  Princeton,  NJ:  Educational  Testing 
Service,  May  1974. 

This  study,  which  is  one  of  a  series  of  ETS  studies  on  the  development  of  measures  of 
cognitive  ability  and  temperament  factors,  was  concerned  with  the  development  and 
evaluation  of  factor  references  or  markers  of  divergent  production.  There  is  little 
evidence  that  the  measures  developed  are  related  to  the  job  performance. 

Ekstrom,  R.  B.,  French,  3.  W.,  &  Harman,  H.  H.  An  attempt  to  confirm  five  recently 
identified  cognitive  factors  (Tech.  Rep.  8).  Princeton,  NJ:  Educational  Testing 
Service,  June  1975. 

This  paper  describes  five  newly-established  cognitive  abilities  (concept  attainment, 
figural  fluency,  integrative  processes,  visual  memory,  and  verbal  closure)  identified  as 
part  of  an  ETS  project  concerned  with  developing  measures  of  abilities  and  temperament 
factors.  Each  of  these  abilities  was  tested  and  analyzed  for  validity. 

Engel,  J.  D.  An  approach  to  standardizing  human  performance  assessment  (HumRRO 
Prof.  Paper  26-70).  Alexandria:  Human  Resources  Research  Organization,  October 
1970. 

Two  areas  of  importance  in  the  standardization  and  evaluation  of  methods  to  assess 
performance  are  task  classification  and  performance  measurement  classification.  This 
report  describes  these  areas  and  reviews  earlier  taxonomic  approaches.  A  four-category 
taxonomy  is  developed  and  an  example  is  presented  which  tries  to  relate  the  task  and 
performance  measurement  classifications. 
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Chambers,  A.  N.  Development  of  a  taxonomy  of  human  performance:  A  heuristic  model 
(Tech.  Rep.  4A).  Washington,  DC:  American  Institutes  for  Research,  October  1969. 

The  paper  begins  with  an  evaluation  of  performance  taxonomies  in  the  behavioral 
sciences.  A  model  for  the  analysis  of  the  issues  involved  in  classifying  human 
performance  is  presented.  The  paper  concludes  with  a  set  of  recommendations  for 
developing  a  behavioral  taxonomy. 

Chenzoff,  A.  P.,  6c  Folley,  3.  D.,  3r.  Guidelines  for  training  situation  analysis  (TSA). 
Valencia,  PA:  Applied  Science  Associates,  1965.  (AD-472  155) 

The  Chenzoff  and  Folley  approach  involves  a  multistage  method  for  task  analysis. 
First  is  a  system  block  analysis,  which  is  a  flow  chart  of  the  major  blocks  of  tasks  or 
system  operations  that  are  all  directed  toward  attaining  a  system  goal.  Then  task-time 
charts  are  prepared  to  identify  each  task  in  a  block,  the  appropriate  operator,  time  for 
completing  events  in  the  block,  time  for  each  task,  and  coordination  required  among  the 
tasks.  Adverse  conditions  affecting  task  performance  are  also  specified.  During  the 
functional  task  description  phase,  the  typical  and  maximal  allowable  times  for  task 
completion  are  identified.  Tasks  are  broken  down  into  major  functional  activities  with 
time  estimates  for  each.  The  behavioral-details  description  identifies  psychological 
characteristics  and  contributes  more  detailed  information.  At  this  stage,  functional 
training  requirements  can  be  defined. 

Cream,  B.  W.,  Eggemeier,  F.  T.,  6c  Klein,  G.  A.  Behavioral  data  in  the  design  of  aircrew 
training  devices.  Proceedings  of  the  Human  Factors  Society's  Nineteenth  Annual 
Meeting,  Dallas,  October  1975. 

This  paper  addresses  the  applicability  of  task  analysis  information  in  the  design  of 
training  devices.  For  relatively  simple  tasks  to  be  trained,  detailed  task  analysis  data  (as 
in  ISD)  are  sufficient  to  design  an  appropriate  trainer.  This  analysis  provides  a  task  list 
with  subtasks,  conditions,  cues,  and  standards  specified.  In  addition,  crew  coordination 
requirements,  if  any,  are  indicated.  For  more  complicated  tasks,  however,  such  task 
analysis  is  not  useful  and  may  be  misleading.  Task  analysis  data  cannot  adequately  define 
these  tasks.  To  develop  a  training  device  for  these  more  complicated  tasks,  one  needs  to 
conduct  indepth  interviews  and  empirical  performance  studies  wherein  cues,  controls,  and 
conditions  are  systematically  altered.  In  this  way,  both  the  necessary  and  sufficient 
trainer  requirements  may  be  identified. 

Dawis,  R.  V.  A  model,  method,  and  taxonomy  for  the  selection  of  tests  as  predictors. 
Minneapolis:  Minneapolis  Civil  Service  Commission,  1977. 

Dawis  describes  a  taxonomy  of  skills  to  be  used  for  employee  selection  and  proposes 
that  for  selection,  skills  should  be  analyzed  into  component  abilities  only.  He  uses  a 
taxonomy  as  a  guide  to  identify  all  the  relevant  abilities  in  each  skill.  As  the  model  is 
meant  to  be  applicable  to  all  job  related  skills  and  abilities,  no  specific  skills  are 
mentioned.  One  of  the  major  advantages  of  this  model  is  that  it  links  performance,  skills, 
and  abilities. 

Demaree,  R.  G.,  6c  Marks,  M.  R.  Development  of  qualitative  and  quantitative  personnel 
requirements  information  (MRI-TDR-62-4).  Wright-Patterson  Air  Force  Base,  OH: 
Behavioral  Science  Laboratory,  1962. 

This  report  describes  the  nature,  purpose,  and  method  for  developing  personnel 
requirements  information.  These  descriptions  form  an  integral  part  of  the  total  system 
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Ammerman,  H.  L.  Systematic  approaches  for  identifying  and  organizing  content  for 
training  programs.  Alexandria:  Human  Resources  Research  Organization,  June  1970. 


Experienced  job  incumbents  are  interviewed  and  asked  to  describe  their  activities  on 
the  job.  A  task  list  is  generated  from  the  responses  of  the  incumbents.  The  key  to  this 
technique  is  the  sophistication  of  the  interviewer.  This  process  is  especially  difficult  to 
apply  to  nonprocedural  jobs  and  jobs  not  directly  oriented  to  hardware  operation  or 
maintenance. 

Austin,  H.  A.  Computational  view  of  the  skill  of  juggling.  Cambridge:  Massachusetts 
Institute  of  Technology,  Artificial  Intelligence  Lab,  1974. 

In  this  study  the  paradigm  used  in  computer  models  of  artificial  intelligence  was 
applied  to  the  physical  skill  of  juggling.  Three  taxonomies  were  developed  on  the  basis  of 
similarity  in  observed  behavior — behaviors,  errors,  correction  of  errors. 

Bernstein,  B.  R.,  &  Gonzalez,  B.  K.  Learning,  retention,  and  transfer  (Tech.  Rep. 
NAVTRADEVCEN  68-C-0215-1).  St.  Paul:  Honeywell  Systems  and  Research  Division, 
September  1969. 

This  study  was  primarily  concerned  with  establishing  research  strategies  in  learning, 
retention,  and  transfer  (L,  R,  and  T)  for  improving  naval  training.  Five  planning  activities 
fed  into  the  formulation  of  the  research  strategy:  (1)  A  brief  review  of  relevant 
literature,  (2)  selection  of  an  appropriate  taxonomy  of  tasks,  (3)  survey  of  Navy  jobs  to 
determine  critical  tasks  from  a  training  standpoint,  (4)  organization  and  management  of 
technical  meeting  with  consultants,  and  (5)  a  conceptual  design  for  a  general  purpose  L, 
R,  and  T  apparatus.  Of  a  secondary  nature,  the  study  also  concerned  itself  with  the 
design  and  conduct  of  two  initial  experiments  to  provide  a  data  base  for  subsequent 
investigations. 

Campbell,  S.  C.,  Feddern,  V.,  Graham,  G.,  &  Morganlander,  M.  A-6E  systems  approach  to 
training.  Orlando:  Naval  Training  Equipment  Center,  February  1977. 

This  report  discusses  the  use  of  mission  scenario  task  lists,  which  are  aids  for 
generating  task  descriptions  specifically  for  the  performance  requirements  of  an  opera¬ 
tional  environment.  A  hierarchy  is  established  for  describing  operator  behaviors  during  a 
typical  mission.  Each  phase  of  a  svitem  mission  is  defined  in  terms  of  major  events,  tasks 
constituting  each  event,  and  the  steps  describing  the  actions  required  by  an  operator  to 
complete  each  task.  Mission  flo-  charts  are  used  to  pictorially  represent  a  mission.  The 
three  levels  of  analysis  are  repre  anted  in  the  mission  flow  chart. 

Caro,  P.  W.  Equipment-device  task  commonality  analysis  and  transfer  of  training  (Tech. 
Rep.  70-7).  Alexandria:  Human  Resources  Research  Organization,  1970.  (AD-709  534) 

Caro  describes  the  stimuli  and  responses  elicited  by  task  elements  and  hardware  in 
the  operational  equipment.  Both  hardware  (displays  and  controls)  and  nonhardware 
(environmental)  stimuli  are  considered.  Parallel  analyses  are  conducted  of  stimulus  and 
response  elements  in  a  training  device.  By  comparing  the  two  lists,  the  critical  stimulus 
components  are  identified  and  predictions  about  device  effectiveness  can  be  made.  Caro 
refers  to  his  method  as  task  commonality  analysis. 
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items  must  approach  the  weapon  system  equipment  in  level  of  fidelity.  By  requesting 
data  limited  to  the  appropriate  breadth  and  depth,  considerable  cost  benefits  can  be 
realized.  The  data  guide  helps  the  project  manager  overcome  these  data-related  problems 
in  a  systematic,  procedural  manner.  Suggested  data  delivery  schedules  presented  key 
delivery  and/or  availability  of  data  to  prime  contract  preliminary  design  reviews  (PDRs) 
or  critical  design  reviews  CDRs)  to  ensure  adequate  maturity  of  data  and  to  minimize  the 
risk  of  changes  to  the  weapon  system. 

Williams,  J.  D.,  Zinke,  R.  A.,  Hardie,  E.  E.,  Kahl,  3.  S.,  Sharpe,  L.  G.,  Sink,  M.  F.,  & 
Vandergaag,  M.  Personnel  subsystem  testing  in  the  man-computer  environment.  A  case 
study.  Proceedings  of  the  Human  Factors  Society  19th  Annual  Meeting,  Dallas,  October 
1975,  186. 

The  development  of  new  man-computer  systems  at  Bell  Telephone  Laboratories 
includes  both  computer  subsystem  and  personnel  subsystem  (PSS)  design  efforts.  There 
has  been  little  information  in  the  literature  to  provide  the  designer  with  guidelines  and 
procedures  for  testing  PSS.  Bell  Laboratoaries  Human  Performance  Technology  Center 
has  developed  a  series  of  testing  procedures  to  provide  useful  performance  data  to  PSS 
designers  and  training  development.  Only  an  abstract  is  provided  in  the  Human  Factors 
proceedings. 

Training  Subsystem 1 

Task  and  Skill  Analysis 

Aerospace  Medical  Division.  Uses  of  task  analysis  in  deriving  training  and  training 
equipment  requirements  (Tech.  Rep.  60-593).  Wright-Patterson  Air  Force  Base,  OH: 
Author,  December  1960. 

This  report  includes  a  series  of  papers  on  task  analysis  presented  at  an  Air  Force 
sponsored  conference.  Descriptions  of  the  following  approaches  to  task  analyses  are 
included:  information-decision-action  chart,  operational  sequence  diagram  (elsewhere 
discussed),  and  task-equipment  analysis.  The  information-decision-action  approach  char¬ 
acterizes  tasks  in  terms  of  information/ decision/ action  units.  Scenarios  are  presented  on 
charts  that  convey  verbal  accounts  of  activities  occurring  during  each  unit.  The  charts 
themselves  portray  tasks  performed  by  an  operator  or  group  of  operators.  If  there  are 
multiple  operators,  relationships  in  time  and  activity  are  depicted.  In  the  task-equipment 
analysis  approach,  tasks,  personnel,  equipment  display,  decisions,  actions,  and  feedback 
are  represented  in  a  matrix  that  allows  the  criticality  of  displayed  information  to  be 
evaluated. 

Amberson,  M.  L.,  Bishop,  D.  D.,  6c  Agoes,  B.  Competency  commonalities  and  accompany¬ 
ing  job  titles  derived  from  the  six  Montana  agriculture  manpower  studies.  Bozeman, 
MT:  Montana  State  University,  Montana  Agricultural  Experimental  Station,  June  1975. 

The  purpose  of  this  project  was  to  determine  the  nature  and  opportunities  in 
agribusiness  and  farming  in  order  to  develop  a  curriculum  that  would  prepare  Montana 
youth  to  meet  these  job  requirements.  The  taxonomy  developed,  which  is  relevant  to 
education  rather  than  to  the  job,  involves  an  undifferentiated  mix  of  skills,  attitudes,  and 
behaviors. 


'This  literature  review  was  facilitated  by  several  existing  literature  reviews  listed  in 
Appendix  B  (e.g.,  Bownas  and  Cooper,  1978;  Chenzoff,  1964;  Dunlap  and  Associates,  Inc. 
1978;  Montemerlo  6c  Tennyson,  1976;  Smith,  1969;  and  Wheaton,  Rose,  Fingerman, 
Karotkin,  &  Holding,  1976). 
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power  trade-off  determination  and  review,  and  ensure  that  manpower/training  informa¬ 
tion  flow  is  timely  and  correctly  channeled. 

Singleton,  3.  W.  The  man-machine  paradigm  in  large  information  processing  systems:  The 
process  of  command  control  system  design.  In  3.  E.  Uhlaner  (Ed.),  Psychological 
research  in  national  defense  today  (Tech.  Rep.  S-l).  Washington,  DC:  U.S.  Army 
Behavioral  Science  Research  Laboratory,  3une  1967. 

The  methodology  applied  to  assure  effective  control  design  in  military  command 
control  systems  is  discussed  and  amplified.  The  three  components  of  a  command  control 
system  must  be  controlled  during  development:  facilities,  hardware,  and  software.  The 
stages  of  the  system  engineering  development  cycle  and  the  output  expected  at  each 
stage  (such  as  the  advanced  planning  stage  that  establishes  objectives  or  the  system 
performance  description  that  achieves  a  functional  description  of  the  operating  system) 
are  reviewed.  A  schematic  diagram  is  presented  that  integrates  the  components  of 
hardware,  software,  and  facilities  with  the  stages  of  the  system  engineering  development 
cycles  to  ensure  that  the  operating  command  control  system  meets  the  information  needs 
of  the  using  military  command. 

Sugarman,  R.  C.,  3ohnson,  S.  L.,  &  Hinton,  W.  M.,  3r.  SAT  revisited — A  critical  post¬ 
examination  of  the  systems  approach  to  training.  Proceedings  of  the  Human  Factors 
Society  19th  Annual  Meeting,  Dallas,  October  1975,  271-273. 

The  systems  approach  to  training  (SAT)  applies  the  techniques  of  systems  analyses  to 
instructional  system  development  to  ensure  the  consideration  of  the  entire  training 
system  including  time  and  external  constraints.  SAT  is  a  conceptual  framework.  A  model 
of  an  information  flow  hierarchy  is  presented.  The  strengths  of  SAT  such  as  relevance, 
interned  documentation,  economy,  and  the  potential  problem  areas,  including  the  ability  of 
the  training  analyst  and  the  need  for  creativity  and  innovation,  are  discussed. 

USAF  3306  Test  and  Evaluation  Squadron,  Mission  Handbook.  Edwards  Air  Force  Base, 
CA:  Author,  3une  1977. 

The  mission  of  this  squadron  is  to  develop  system-specific  maintenance  training 
programs  and  training  equipment  requirements.  This  mission  is  achieved  by  implementing 
instructional  system  development  (ISD)  methods.  The  particular  way  these  methods  are 
applied  by  the  3306th  is  the  subject  of  the  handbook.  The  procedures  described  are 
intended  to  maximize  objectivity  in  decisions  related  to  program  development  and 
derivation  of  equipment  requirements.  This  is  achieved  through  extensive  use  of 
procedural  checklists  and  decision  trees.  Full  documentation  of  decision  factors  ensures 
adequate  justification  of  recommendations.  The  handbook  is  not  a  cookbook,  but  a  means 
of  reducing  subjectivity  and  providing  a  common  basis  for  decisions. 

U.S.  Army  Project  Manager  for  Training  Devices  (PM  TRADE).  Project  Manager's 
Training  Device  Data  Guide,  Volume  1.  Orlando:  Author,  September  1978. 

The  data  guide  was  developed  to  provide  weapon  system  project  managers  with  a 
method  for  facilitating  concurrent  development  of  the  weapon  system  and  training 
devicefe).  Two  major  problems  associated  with  concurrent  development  are  the  inability 
to  specify  the  type  and  amount  of  data  necessary  for  training  device  development  and  the 
lack  of  contract  procedures  for  ensuring  that  data  are  provided  to  the  training  device 
contractor  in  a  timely  manner.  Data  can  be  limited  in  two  ways:  in  breadth  by  requesting 
data  for  only  those  weapon  system  equipment  items  to  be  represented  in  the  training 
device,  and  in  depth  by  determining  the  degree  to  which  the  training  device  equipment 


4.  Identification  of  potential  perturbing  events  (literature  search,  interview  data, 
contractor  base,  intuition  of  study  team). 

5.  Categorization  of  events  by  probability,  according  to  the  judgment  of  survey 
participants  (queried  by  questionnaire). 

6.  Examination  of  potential  effect  of  probable  events  upon  extrapolated  trends  to 
produce  a  surprise-free  scenario  of  the  NAVAIR  environment  1985-1995  (interview  data, 
panel  discussions,  structured  workshops,  judgment  of  study  team). 

Neiswender,  L.  Maximizing  your  training  efficiency;  The  application  of  behavioral 


principles  to  job  training.  Washington,  DC;  U.S.  Department  of  Labor  Manpower 
Administration,  November  1972. 

This  training  manual  is  designed  to  instruct  work  supervisors  and  skills  trainers  of  the 
hard-to-employ  disadvantaged  in  the  effective  use  of  behavioral  reinforcement  principles 
for  overcoming  difficult,  frequently  encountered  training  problems.  One  of  its  primary 
focuses  is  how  to  train  for  job-required  behavioral  skills  such  as  punctuality  or  taking 
criticism. 

Parsons,  S.  O.,  &  Lebach,  3.  L.  The  standardization  of  human  factors  data.  Human 
Factors,  1970,  12,  55-62. 

Although  the  success  of  the  Apollo  Applications  Program  requires  the  close  integra¬ 
tion  of  human  factors  documentation,  human  factors  data  are  developed  piecemeal  in  its 
early  phases.  The  objective  of  this  study  was  to  develop  standard  human  factors  reporting 
techniques  compatible  with  computer  data  processing  methods.  Thirty-two  human  factors 
data  forms  and  techniques  were  analyzed.  A  matrix  was  developed  that  organizes  17 
document  and  data  types  by  43  specific  content  areas.  Representatives  from  the  six  user 
organizations  indicated  the  needs  of  their  organization  in  performing  specific  design  and 
development  functions.  High  frequency  data  items  formed  the  basis  for  developing  a 
single  format  that  can  be  applied  to  any  new  manned  system. 

Porter,  E.  H.  A  paradigm  for  system  analysis  of  command  and  control  functions.  In  J.  E. 
Uhlaner  (Ed.),  Psychological  research  in  national  defense  today  (Tech.  Rep.  S-l). 
Washington,  DC;  U.S.  Army  Behavioral  Science  Research  Laboratory,  3une  1967. 

The  traditional  stimulus  response  paradigm  is  discussed.  A  model  is  illustrated  that 
outlines  the  functional  relationships  and  dynamic  interrelationships  found  in  most  systems, 
and  applies  this  model  to  an  air-defense  command  control  system. 

Ruland,  3.  K.,  Akman,  A.,  Ettlinger,  L.,  Kramer,  H.,  Luhring,  R.,  <5c  Peterson,  F.  Military 
manpower  versus  hardware  procurement  study  (HARDMAN).  Washington,  DC;  Office 
of  the  Chief  of  Naval  Operations,  February  1977. 

This  is  the  final  report  of  a  10-month  study  to  analyze  the  definition  of  manpower- - 
training  requirements  during  the  Weapons  System  Acquisition  Process  (WSAP).  The 
authors  conclude  that  manpower/ training  considerations  occur  too  late  in  the  WSAP  and, 
in  many  cases,  receive  too  little  attention  relative  to  hardware  development  effort. 
Recommendations  to  improve  this  situation  so  that  manpower/training  factors  would 
receive  adequate  consideration  at  appropriate  times  in  the  WSAP  include;  Establish  a 
HARDMAN  Project  Office  to  oversee  addressing  of  manpower/training  issues  appropriate¬ 
ly,  develop  manpower/ training  forecasting  models  to  assist  in  early  requirements  defini¬ 
tion,  implement  man  power/ training  cost  and  trade-off  models  to  assist  in  hardware/man 
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conclusion  that  ISD  approaches  cannot  be  successfully  used  by  the  layman  to  replace  an 
experienced  training  development  analyst.  He  suggests  that  further  research  emphasize 
development  of  ways  to  assist  rather  than  replace  the  expert  training  developer. 

Naval  Weapons  Engineering  Support  Activity.  A  prediction  of  aviation  logistics  require¬ 
ments  for  the  decade  1985-1995.  Washington,  DC:  Author,  June  1978. 

Methodology  used  to  forecast  Navy  logistical  support  rests  heavily  on  the  "informed 
judgments"  of  200  representatives  from  industry  and  government.  The  following  questions 
are  addressed: 

1.  What  do  we  want  our  position  to  be  in  the  year  X? 

2.  What  will  our  position  be  in  the  year  X  if  present  trends  continue? 

3.  What  is  likely  to  happen  before  the  year  X  to  perturb  present  trends,  and  what 
will  be  the  effect? 

4.  What  discrepancies  exist  between  this  probable  future  and  the  desired  one? 

5.  What  options  do  we  have  which  may  reduce  these  discrepancies? 

6.  What  policy  can  we  follow  to  enhance  the  probability  of  achieving  our  desired 
position  in  the  year  X? 

All  the  many  various  forecasting  techniques  are  designed  to  address  one  or  more  of 
these  questions.  In  the  jargon  of  technology  forecasters,  the  answer  to  question  1 
constitutes  a  normative  forecast;  question  2  results  in  a  baseline  scenario;  question  3 
identifies  external  events  that  will  act  upon  the  baseline  scenario  to  produce  a  surprise- 
free  scenario.  The  gap  analysis  conducted  in  response  to  question  4  is  followed  by  the 
exploration  of  alternative  scenarios  consequent  upon  the  occurrence  of  other  planned/un¬ 
planned  events  (question  5)  to  determine  their  effects  upon  the  identified  discrepancies. 
With  these  data,  policy  makers  can  make  an  informed  selection  of  the  available  options 
(resource  allocation,  mission  formulation,  equipment  buys,  etc.)  (question  6)  to  maximize 
the  probability  of  achieving  their  goals. 

In  addition  to  the  individual  interviews,  the  following  additional  activities  were  used: 
problem  definition/analysis,  literature  search,  data  analysis,  structural  questionnaire 
soliciting  estimates  of  key-event  probabilities  distributed  to  informed  military/industrial 
personnel,  general  panel  discussions  with  selected  expert  military/industrial  personnel, 
panel  data  analysis,  structural  impact  prediction  expercise  with  selected  expert  mili¬ 
tary/industrial  personnel,  data  synthesis  and  formulation  of  findings,  distribution  of 
findings,  and  continuing  review,  analysis  and  refinement  of  findings  by  study  team. 

Initial  intuitive  conclusions  were  verified  using  the  following  established  techniques 
of  technological  forecasting: 

1.  Identification  of  a  set  of  descriptors  (i.e.,  economic  and  technologic  trends 
descriptive  of  the  NAVAIR  environment)  as  a  result  of  the  literature  search,  interview 
summaries,  and  the  judgments  of  the  study  team. 

2.  Cumulation  of  historic  values  for  these  trends  over  as  long  a  time  period  as 
possible  (literature  search,  contractor  data  base,  personal  discussions). 

3.  Mathematical  extrapolation  of  these  trends  through  1995. 
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Maintenance  Training  (Transforming  High  School  Graduates  into  Sonar  Engineers). 
Training  for  sonar  maintenance  has  received  greater  attention  than  training  for  sonar 
operations.  Nonetheless  it  is  doubtful  whether  the  average  crew  of  technicians  can  keep 
today's  sonar  systems  in  peak  operating  condition.  It  may  be  necessary  to  develop 
technician-engineers  who  can  solve  maintenance  problems  on  a  systems  level  for  complex 
systems  such  as  the  SQS-26  and  BQ5-13  sonars.  The  report  lists  related  recommendations 
from  the  previous  ASWAC  study. 

ASW  System  Design  and  Human  Engineering  Considerations.  The  trend  toward 
designing  ever  more  complex  and  large  sonar  systems  has  resulted  in  a  corresponding 
inclination  of  reducing  the  role  of  personnel  to  simple  monitors  of  automatic  systems. 
The  report  recommends  that  greater  attention  be  devoted  to  the  trade-offs  in  function 
allocation  between  man  and  machine.  Where  automatic  functions  are  introduced,  they 
should  be  clearly  related  to  the  most  likely  areas  of  human  failure.  The  needs  of  the 
system  should  be  more  carefully  analyzed  and  assessed  from  the  standpoint  of  operator 
use  and  effectiveness.  A  data  base  should  be  developed  for  maintaining  a  historical 
appraisal  of  the  capabilities  and  deficiencies  of  sonar  systems.  Related  recommendations 
from  a  previous  ASWAC  study  are  listed  and  evaluated. 

Miller,  L.  A.,  McAleese,  K.  J.,  Erickson,  3.  M.,  Klein,  G.  A.,  &  Boff,  K.  R.  Training 
device  design  guide:  The  use  of  training  requirements  in  simulation  design  (F33615-76- 
C-0050).  Wright-Patterson  Air  Force  Base,  OH:  Air  Force  Human  Resources 
Laboratory,  June  1977. 

This  guide  provides  a  process  for  the  design  of  trainers  based  on  behavioral 
information.  Behavioral  information  includes  task  analyses,  training  requirements, 
training  scenarios,  perceptual  (display/control)  requirements,  malfunction/problem  scen¬ 
arios,  and  other  information  that  has  freqently  been  ignored  when  developing  simulation 
designs.  The  guide  outlines  the  information  requirements  of  trainer  development 
personnel  and  presents  a  series  of  guidelines  (e.g.,  checklists,  flowcharts,  and  examples). 
Simulation  fidelity  and  instructional  features  of  trainers  ai  considered  in  some  detail. 
The  use  of  a  design  team  to  ensure  that  appropriate  people  get  the  information  needed  to 
make  the  right  decisions  is  recommended.  This  guide  is  intended  as  an  aid  to  trainer 
development  and  not  an  algorithm. 

Montemerlo,  M.  D.  Instructional  systems  development  state  of  the  art  and  directions  for 
the  future.  Proceedings  of  the  8th  NTEC/Industry  Conference,  Orlando,  November 
1975,  421-429. 

Instructional  systems  development  (IS D)  approaches  may  be  seen  to  be  models  of 
expert  training  program  developers.  The  paper  investigates  the  extent  to  which  ISD 
methods  in  general  can  be  used  by  laymen  to  replace  the  expert  developer.  The  success  of 
ISD  is  related  to  the  degree  that  procedures  for  selecting  and  implementing  optimal 
instruction  can  be  devised.  After  a  literature  review,  the  conclusion  is  that  there  is  no 
theoretical  basis  for  development  of  algorithms  to  generate  instructional  programs. 
Examining  the  experiences  of  those  who  have  attempted  use  of  ISD,  Montemerlo  found 
that  personnel  require  some  training  to  apply  ISD  correctly.  Close  supervision  by  an 
experienced  training  developer  was  also  required  in  some  cases.  It  is  unrealistic  to  think 
that  one  ISD  model  can  solve  all  training  development  problems,  since  large  variations  of 
ISD  methodology  are  required  by  different  content  areas.  Current  ISD  approaches  are 
utilized  for  not  including  all  the  necessary  steps  and  for  not  including  sufficient  detail  to 
perform  some  steps.  Also,  ISD  models  fail  to  specify  skill  and  knowledge  required  of  the 
ISD  user.  Some  of  the  political,  innovative,  creative,  managerial  skills  and  technical 
knowledge  that  seems  to  be  required  are  suggested.  These  considerations  lead  to  the 


to  "top  drawings”  of  training  program  sequences  in  an  engineering  fashion.  Finally,  a 
training  control  monitors  the  training  development/production  activity  in  much  the  same 
way  as  an  engineering  change  control  would  tn  incorporate  updates  oi  requirements  and 
scheduling.  Although  some  issues  still  need  to  resolved  (e.g.,  skill  level  specifications 
and  media  selection  criteria),  the  author  maintains  that  the  approach  works  and  that 
standardizing  and  cataloging  of  behavior  should  begin. 

Mackie,  R.  R.  Personnel,  training  and  human  factor  problems  relating  to  ASW  effective¬ 
ness.  Goleta,  CAs  Human  Factors  Research,  February  1970. 

The  objective  of  this  study  was  to  examine  the  impact  of  the  Navy's  current  policies 
on  ASW  systems  performance  with  respect  to  the  selection,  training,  and  utilization  of 
ASW  personnel,  and  from  the  standpoint  of  the  design  of  the  ASW  systems  for  human  use. 
Recent  trends  in  training  and  systems  design  are  emphasized.  Each  of  the  five  main 
sections  in  this  report  deals  with  a  major  problem  area. 

Career  Pattern  and  Training  of  the  ASW  Officer.  The  U.S.  Navy  still  depends 
primarily  on  the  "well-rounded"  general  purpose  line  officer  for  the  operation,  mainten¬ 
ance,  and  tactical  employment  of  the  most  complex  ASW  systems  ever  developed.  There 
appears  to  be  no  clear  evidence  of  the  Navy  encouraging  a  career  pattern  that  might  lead 
to  ASW  specialists.  Only  8  weeks  of  formal  training  are  given  to  ASW  officers,  the 
majority  of  whom  have  no  shipboard  experience.  Many  students  lack  technical  back¬ 
ground.  This  report  recommends  that  the  Chief  of  Naval  Personnel  develop  a  career 
pattern  for  ASW  officer  specialists  and  conduct  a  detailed  study  of  the  knowledge  and 
performance  requirements  of  officers  with  ASW  responsibilities  to  determine  the  depth  of 
instruction  required  for  the  effective  performance  by  officers  in  all  stated 
responsibilities.  It  records  and  evaluates  related  recommendations  from  a  previous 
ASWAC  study. 

Selection  and  Retention  of  ASW  Personnel.  This  section  includes  the  results  of  a 
Navy  survey  of  the  motivational  factors  influencing  career  decisions  for  the  military 
source.  The  conclusions  of  the  "Pride  Board"  report  (OPNAV  NOTICE  3500,  OP-09B83, 
Ser  3191P09B8,  dated  18  December  1961)  are  still  valid.  This  report  recommends  that  (1) 
an  ASW  officer  career  ladder  be  established  that  ensures  promotion  on  the  basis  of 
demonstrated  competence  in  ASW;  (2)  ASW  departments  be  established  aboard  all  ships 
having  ASW  as  their  primary  mission  and  the  secondary  m.ssions  of  ASW  ships  and  aircraft 
be  limited;  and  (3)  all  ASW  teams  be  maintained  as  integral  units  at  least  until  minimum 
standards  of  readiness  have  been  demonstrated.  Six  recommendations  from  the  previous 
ASW  study  are  reported  and  evaluated. 

ASW  Team  Training  (Questions  of  When  and  Where).  Since  WWII,  there  has  been  a 
pattern  to  reduce  training  at  sea  and  increase  training  ashore  using  ASW  simulators. 
Operational  training,  for  the  purposes  of  this  report,  includes  both  the  training  of 
individual  ASW  personnel  and  of  the  ASW  team.  The  most  complex  and  demanding  tasks 
confronting  ASW  sensor  operators  are  those  relating  to  target  detection  and  classifica¬ 
tion.  These  are  the  areas  in  which  shore-based  training  is  weakest.  The  report  supports 
efforts  directed  toward  the  definition  of  operational  training  requirements  and  operation¬ 
al  training  devices  for  all  systems  with  a  reasonably  long  life  expectancy.  It  recommends 
that  the  opportunities  for  practice  in  operating  techniques  and  realistic  detection  and 
classification  capabilities  be  included  in  shore-based  team  trainers.  It  recommends 
defining  the  responsibilities  of  the  school  and  the  fleet  clearly  for  each  training 
requirement.  It  reports  and  evaluates  related  recommendations  from  a  previous  ASWAC 
study. 
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After  the  training  environment  has  been  specified  for  each  training  objective,  a 
syllabus  is  generated.  Each  syllabus  block  is  specified  as  a  set  of  training  objectives  to  be 
taught  in  a  common  time  period  in  the  same  training  device.  The  specific  environmental 
characteristics  necessary  to  train  the  syllabus  block  can  then  be  determined  by  adding  up 
the  environmental  characteristics  for  each  training  objective.  Blocks  that  require  similar 
training  environments  can  be  defined  as  involving  the  same  training  device.  The  essential 
environmental  characteristics  of  the  training  device  can  then  be  translated  into  functional 
specifications  with  the  assistance  of  training  device  designers. 

This  approach  is  intended  to  ensure  that  the  essential  environments  in  which  the 
student  must  be  trained,  exist  in  at  least  one  device.  Additionally,  only  those 
environments  that  are  necessary  will  be  identified.  Finally,  a  trade-off  analysis  of  device 
features  between  different  training  devices  can  be  conducted  to  ensure  that  each  device 
is  adequately  utilized.  A  potential  criticism  of  this  procedure  is  its  reliance  on  expert 
judgment  to  determine  the  necessary  environmental  characteristics  for  each  training 
objective.  The  stated  advantage  of  this  approach  is  that  these  judgments  are  being 
exercised  on  relatively  small  training  units,  as  opposed  to  decisions  based  upon  the 
training  of  the  overall  task.  This  approach  also  recognizes  the  strong  interdependencies 
between  course  design  and  resource  requirements. 

King,  W.  3.,  (c  Tennyson,  M.  E.  Maintenance  training  and  aiding  in  the  1980's:  A 
prediction.  Proceedings  of  the  First  International  Learning  Technology  Congress  and 
Exposition  on  Applied  Learning  Technology,  Volume  IV,  Washington,  DC,  July  1976,  40- 
43. 

At  present,  a  significant  portion  of  the  DoD  budget  is  allocated  to  maintenance 
training;  at  the  same  time,  little  is  spent  on  research  to  improve  the  quality  of  this 
training  despite  the  growth  of  increasingly  complex  equipment  to  be  maintained.  The 
paper  focuses  on  using  simulation  and  aiding  for  improving  the  maintenance  of  future 
weapon  systems.  In  considering  current  aspects  of  training  and  aiding,  it  concludes  that 
some  form  of  mixing  of  the  two,  "traiding,"  would  need  to  be  continued  into  the 
foreseeable  future.  The  Naval  Training  Equipment  Center  concept  of  modular  integration 
of  training  information  by  a  performance  aiding  computer  (MITIPAC)  is  seen  to  offer  a 
solution  for  future  needs.  Although  the  use  of  automated  test  equipment  in  the  future 
modifies  the  nature  of  maintenance,  maintenance  training  is  still  a  major  consideration. 
Finally,  implications  of  the  Navy  Enlisted  Occupation  Classification  System  (NEOCS)  are 
examined.  According  to  NEOCS,  major  individual  training  expenses  would  be  deferred 
until  after  the  individual  has  made  a  commitment  to  remain  in  the  Navy  for  an  extended 
time.  Exportable,  individualized  training  via  MITIPAC  along  with  on-the-job  training  are 
considered  as  viable  ways  to  obtain  productive  output  without  prior  expensive  training 
courses. 


Klemmer,  A.  Training:  An  engineering  process.  Proceedings  of  the  Sixth  Naval  Training 
Equipment  Center  and  Industry  Conference,  Orlando,  November  1973,  449-459.  " 

The  author  argues  for  adoption  of  an  engineering  approach  to  the  development  and 
management  of  training  programs.  To  indicate  the  feasibility  of  this,  the  approach  taken 
by  Douglas  Aircraft  Company's  Military  Training  Department  is  briefly  described.  It  is 
pointed  out  that  the  procedures  used  by  Douglas  parallel  engineering  hardware  develop¬ 
ment  activities.  Key  to  the  training  program  is  the  establishment  of  Training  Instruction 
Requirement  Specifications  (TIRS)  numbers  which  course  planners  would  use  as  engineers 
would  use  part  numbers  to  design  a  piece  of  equipment.  TIRS  numbers  specify:  method  of 
instruction,  media  to  be  used,  prerequisite  instruction,  time  estimate,  instructional 
objectives  for  course  planning,  and  student  behavioral  objectives.  These  TIRS  are  applied 


Merrill,  P.  F.  Task  analysis;  An  information  processing  approach  (Tech.  Memo.  27). 
Washington,  DC:  Office  of  Naval  Research,  April  1971. 

This  study  proposes  and  describes  an  information  processing  approach  to  task 
analysis.  A  learning  task  may  be  analyzed  at  two  levels:  a  task  level  concerned  with 
subskills  required  to  learn  and  perform  a  terminal  objective,  and  a  rule  level  entailing  the 
most  effective  instructional  method  for  teaching  subskills.  This  report  focuses  on  the 
task  level.  It  contrasts  information  processing  and  hierarchical  task  analyses  models  and 
concludes  that  neither  approach  is  sufficient  for  all  tasks.  Hierarchical  analysis  is  more 
appropriate  for  lower  ordered  skills,  while  information  processing  analysis  should  be  used 
when  the  output  of  one  task  subskill  is  required  input  for  a  succeeding  operation. 

Meyer,  R.,  Laveson,  J.,  Weissman,  N.,  &  Eddowes,  E.  Behavioral  taxonomy  of  undergradu¬ 
ate  pilot  training  tasks  and  skills:  Taxonomy  refinement,  validation,  and  operation. 
St.  Louis:  Design  Plus,  December  1974. 

This  report  describes  a  program  concerned  with  analyzing  the  fundamental  tasks 
involved  in  undergraduate  pilot  training.  The  goals  of  the  program  were  to  determine  the 
flying  skills  and  behavioral  elements  involved  in  flying  and  to  develop  a  taxonomy  relating 
tasks,  skills,  and  behavioral  elements.  Flying  tasks  are  analyzed  into  one  of  three  types: 
fundamental,  composite,  or  continuous.  Because  the  classification  system  is  easy  to  use 
and  requires  only  a  knowledge  of  flying,  it  shows  great  promise  for  use  in  training  and 
evaluation.  The  approach  used  in  developing  this  taxonomy  is  more  sophisticated  than 
that  used  in  most  taxonomies. 


Miller,  R.  B.  A  method  for  man-machine  task  analysis  (Tech.  Rep.  53-137).  Pittsburgh: 

The  American  Institute  for  Research,  3une  1953. 

An  operational  situation  is  broken  down  into  missions.  For  each  mission  a  time  chart 
is  prepared  that  lists  tasks  and  then  groups  them  as  functions  of  time,  similarity  of  skill, 
or  equipment  requirements.  Both  time  continuity  and  time  sharing  are  considered.  The 
procedure  is  as  follows.  In  the  first  step,  gross  tasks  are  identified  and  the  input  features 
of  each  task  are  analyzed  by  addressing  task-relevant  cues  for  initiating  task  perform¬ 
ance,  stimulus  differences  for  discrimination,  alternative  choices,  and  characteristic 
errors  for  malfunctions  associated  with  the  task.  The  second  step  involves  the  analysis  of 
subtasks  with  emphasis  on  the  response  aspects  of  a  task  and  addresses  similar  features, 
as  well  as  identifying  feedback  mechanisms  and  objective  criteria  for  assessing  adequacy 
of  response. 

Miller  also  addressed  the  task  analysis  of  continuous  feedback  tasks.  He  describes 
the  skills  involved  in  making  continuous  adjustments  to  changing  stimulus  conditions  and 
reactions  to  operator  adjustments.  Attention  is  focused  on  controls,  displays,  and  the 
feedback  mechanisms. 

Miller,  R.  B.  A  method  for  determining  task  strategies  (Tech.  Rep.  AFHRL-IR-74-26). 

Pittsburgh:  American  Institutes  for  Research,  May  1974. 

Ths  study  is  an  extension  of  Miller's  task  analysis  methodology.  The  concept  of 
strategic  principles  or  task  strategies  implicit  in  the  job  activities  of  highly  proficient 
performers  is  examined  with  the  intent  of  improving  training.  The  basic  approach 
considers  the  human  performer  as  an  information  processing  system  and  describes  unique 
features  of  skilled  performance  in  information  processing  terms.  A  set  of  25  terms 
associated  with  information  processing  are  discussed.  The  set  can  more  or  less  be  grouped 
under  the  functions  of  input-processing-memory-output.  A  procedure  for  detecting  or 


inventing  strategies  that  distinguish  skilled  from  unskilled  performance  of  a  given  task  is 
outlined.  In  general,  the  process  involves  analyzing  the  job  (including  the  general  goal  and 
major  steps)  in  transaction  (information  processing)  terms,  and  after  some  simplification, 
creating  and  testing  a  strategy  that  most  adequately  fulfills  the  criteria  for  the 
accomplishment  of  the  goal  or  mission. 

Mitchell,  3.  F.,  6c  Ramsey,  T.  A.  B-l  systems  approach  to  training  task  analysis  listings. 
Buffalo:  Calspan  Corporation,  3uly  1975. 

Calspan  developed  a  hierarchical  method  for  organizing  system  information.  The 
format  of  the  task  analysis  data  is  initiation  cue,  action  verb,  control,  and  completion 
cue.  The  initiation  cue  consists  of  a  combination  of  relational  statements  involving 
controls  or  displays,  a  connective,  and  value  or  state.  The  action  verbs  are  standardized 
terms  corresponding  to  the  control  that  is  operated  upon.  The  control  is  the  direct  object 
of  the  action  verb.  In  some  cases  a  display  can  be  operated  on  and  becomes  the  control. 
The  completion  cue,  which  indicates  the  termination  of  a  behavioral  unit,  may  serve  as 
the  intiating  cue  for  the  next  behavioral  unit.  Each  behavioral  unit  is  called  a  task 
element.  Task  elements  are  grouped  to  form  tasks,  which  in  turn  are  grouped  to  form 
functions.  Mission  segments  are  defined  as  groups  of  functions.  Computer  storage  and 
retrieval  are  utilized  in  the  Calspan  approach. 

Montemerlo,  M.,  6c  Eddowes,  E.  The  judgmental  nature  of  task  analysis.  Proceedings  of 
the  Human  Factors  Society  22nd  Annual  Meeting,  Detroit,  October  1978,  247-250. 

The  notion  that  a  best  task  analysis  procedure  exists  is  examined  critically.  There 
are  many  tasks  for  which  the  best  way  of  doing  them  cannot  be  described.  Other  tasks, 
however,  can  be  performed  in  a  best  way.  Task  analysis  appears  to  be  a  task  for  which 
the  best  way  it  should  be  performed  cannot  be  described.  It  is  difficult  to  find  two 
instructional  technologists  who  agree  on  how  to  conduct  a  task  analysis.  Also,  there  are 
persons  who  can  perform  a  good  task  analysis  and  others  who  cannot.  The  process  of  task 
analysis  has  not  been  amenable  to  task  analysis. 

The  ability  to  proceduralize  is  inversely  proportional  to  the  degree  to  which  a  task  is 
affected  by  other  variables.  This  ability  is  also  inversely  related  to  the  extent  that  one 
step  influences  others.  If  problem-solving  behavior  is  involved,  the  task  is  difficult  to 
proceduralize.  The  size  of  the  domain  of  possible  task  behaviors  also  affects  the  ability 
to  proceduralize  a  task.  Interpersonal  skills  are  hard  to  proceduralize  (task  analysis  itself 
is  an  interpersonal  skill).  Tasks  involving  cognitive  skills  are  difficult  to  proceduralize, 
because  they  are  hard  to  make  observable.  Task  analysis  is  highly  loaded  with  cognitive 
skills.  The  scientific  basis  for  task  analysis  is  poor  and,  therefore,  measures  for  task 
analysis  are  not  well  developed. 

These  factors  indicate  that  task  analysis  is  not  reducible  to  a  well-defined  procedure. 
The  historical  way  to  learn  a  judgmental  task,  such  as  task  analysis,  is  via  apprenticeship 
training.  Recently,  the  military  has  begun  to  be  aware  of  the  judgmental  nature  of  task 
analysis  and  the  fruitlessness  of  attempts  at  its  proceduralization. 

Munger,  M.  R.  The  study  of  training  equipment  requirements  for  tactical  data  systems, 
project  memorandum  //2:  A  systems  approach  to  the  determination  of  functional 
training  equipment  requirements.  Pittsburgh:  American  Institute  for  Research, 
November  1959. 

Munger's  approach  includes  six  major  techniques:  functional  analysis  (allocation  of 
functions  between  man  and  machine  and  identification  of  gross  tasks);  skill  and  knowledge 


analysis  (determination  of  primary  skill  and  knowledge  requirements);  task  equipment 
analysis  (specification  of  procedure  and  equipment),  error  analysis  (identification  of 
potential  human  errors);  mission  analysis  (process  of  analyzing  system  missions);  and 
contingency  analysis  (listing  of  nonroutine  situations).  This  information  is  used  in 
identifying  tasks,  determining  stimulus- response  features,  identifying  possible  response 
errors,  and  specifying  required  training  programs. 

Parker,  J.  F.,  Jr.,  6c.  Downs,  J.  E.  Selection  of  training  media  (Tech.  Rep.  ADS  TR  61- 
473).  Wright-Patterson  Air  Force  Base,  OH:  Aeronautical  Systems  Division,  September 
1961. 

Parker  and  Downs  discussed  the  translation  of  statements  of  desired  personnel 
performances  and  capabilities  and  task  analysis  documents  into  training  objectives.  They 
identify  three  classes  of  simulation  of  the  contextual  environment.  Extraneous  contextual 
cues  include  general  irrelevant-environment  cues  accompanying  task  performance.  Per¬ 
formance  degraders  are  forces  that  cannot  be  overcome  to  any  appreciable  extent  by 
practice  and  tend  to  reduce  performance  capabilities  in  a  relatively  constant  manner. 
Performance  disruptors  are  forces  that  are  disruptive  at  first,  but  can  be  overcome 
through  practice.  Therefore,  while  performance  disruptors  should  be  simulated,  there  is 
little  benefit  to  simulating  performance  degraders. 

Powers,  T.  E.  Selecting  presentation  modes  according  to  personnel  characteristics  and 
the  nature  of  job  tasks.  Baltimore  County:  University  of  Maryland,  January  1977. 

The  purpose  of  the  study  was  to  improve  Navy  technical  manuals  by  finding  a  way  to 
match  user  characteristics  with  manual  characteristics.  Although  this  report  contains  an 
extensive  review  of  the  literature  on  behavioral  and  task  taxonomies,  it  is  mainly 
concerned  with  the  development  of  a  questionnaire  to  measure  the  behaviors  required  for 
a  number  of  technical  jobs.  The  questionnaire  was  administered  to  452  petty  officers.  Its 
results  were  used  to  develop  a  taxonomy  of  cognitive  behaviors  for  technical  jobs  and  to 
determine  what  information  is  appropriate  to  include  in  training  manuals. 

Rankin,  W.  C.  Task  description  and  analysis  for  training  system  design  (Tech.  Memo  74- 
2).  Orlando:  Navy  Training  Analysis  and  Evaluation  Group,  January  1975. 

This  report  examines  what  constitutes  task  description  and  task  analysis  in  the 
context  of  training  system  design.  Task  description  is  a  physical  description  of  tasks  in 
the  form  of  a  task  inventory.  Task  analysis  is  a  behavioraliy  analytic  description.  The 
advantages  and  disadvantages  of  two  approaches  to  task  descriptive  data  collection — the 
interview  and  the  questionnaire- -are  discussed.  Selected  current  efforts  in  task  descrip¬ 
tion  in  the  naval  training  community  are  reviewed,  but  not  evaluated  (ILDG,  PQS, 
NOTAP,  TAEG).  The  TAEG  approach  to  task  description  is  presented  in  an  appendix, 
which  includes  an  instrument  (forms)  for  collecting  task  information  and  a  primer  of 
instructions  for  completing  them. 

Riccobono,  J.  A.,  6c  Cunningham,  J.  W.  Work  dimensions  derived  through  systematic  job 
analysis:  A  replicated  study  of  the  occupational  analysis  inventory.  Raleigh:  North 
Carolina  State  University,  Center  for  Occupational  Research,  1971. 

This  is  part  of  a  series  of  studies  concerning  the  occupational  analysis  inventory 
(OAI),  which  is  a  quantitative  system  for  describing  and  measuring  work.  The  present 
study  was  to  test  the  stability  of  the  factors  found  in  prior  research  with  a  sample  of  400 
new  jobs. 
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Shettel,  H.  H.,  Sc  Horner,  W.  R.  Functional  requirements  for  driver  training  devices 
(Tech.  Rep.  86400  10/72-FR).  Pittsburgh:  American  Institute  for  Research,  1972. 

This  approach  identifies  key  learning  elements,  categorizes  them  according  to  type  of 
behavior,  identifies  relevant  training  techniques,  and  arranges  these  techniques  into 
training  programs.  Task  analysis,  which  is  addressed  at  a  middle  level  (i.e.,  not  too  global 
or  too  specific),  mainly  identifies  major  behavioral  events.  For  each  task  a  definition  of 
the  task,  including  initiating  cues  and  terminating  events,  and  statement  of  prerequisite 
skills  is  provided.  Difficult-to-learn  or  highly  critical  elements  are  also  identified.  An 
important  addition  is  a  rationale  for  training.  From  this  information,  appropriate 
strategies  and  techniques  are  identified.  Further  definition  of  learning  objective,  desired 
response,  and  feedback  requirements  are  part  of  the  overall  analysis. 

Siegel,  A.  I.,  <5c  Federman,  P.  J.  Increasing  ASW  helicopter  effectiveness  through 
communications  training  (Tech.  Rep.  NAVTRADEVCEN  66-C-004501).  Orlando:  Naval 
Training  Device  Center,  October  1978. 

Siegel  and  Federman  used  an  "amount  of  intellectual  operation"  scale  to  rate  the 
contribution  of  cognitive,  memory  divergent  production,  convergent  production,  and 
evaluation  operations  in  the  performance  of  a  particular  task.  The  method  indicates 
which  tasks  impose  the  heaviest  intellectual  load  and,  therefore,  require  training.  It  also 
identifies  the  specific  types  of  intellectual  activity  required.  The  approach  is  based  on 
J.  P.  Guilford's  structure  of  intellect  model. 

Theologus,  G.  C.,  Sc  Fleishman,  E.  A.  Development  of  a  taxonomy  of  human  performance: 
Validation  study  of  ability  scales  for  classifying  human  tasks.  Washington,  DC: 
American  Institutes  for  Research,  1971. 

This  is  a  technical  report  of  the  AIR  project  to  develop  a  human  performance 
taxonomy.  The  Task  Assessment  Scales  (TAS)  were  developed  to  operationalize  the 
ability  requirements  approach  to  performance  assessment.  The  scales  of  the  TAS  were 
tested  both  for  construct  validity  and  predictive  validity. 

Theologus,  G.  C.,  Romashko,  T.,  Sc  Fleishman,  E.  A.  Development  of  a  taxonomy  of 
human  performance:  A  feasibility  study  of  ability  dimensions  for  classifying  human 
tasks.  Washington,  DC:  American  Institutes  for  Research,  1970. 

This  report  describes  the  initial  development  of  the  Task  Assessment  Scales  (TAS),  a 
classification  system  based  on  ability  requirements  for  performance. 

Tornow,  W.  W.,  Sc  Pinto,  P.  R.  The  development  of  a  managerial  job  taxonomy:  A  system 
for  describing,  classifying,  and  evaluating  executive  positions.  Journal  of  Applied 
Psychology,  1976,  61,  410-418. 

This  study  attempted  to  replicate  and  improve  Hemphill's  research  on  managerial 
jobs.  A  questionnaire  was  developed  to  include  items  concerning  managerial  activities, 
concerns,  responsibilities  and  demands,  and  tested  empirically.  This  is  one  of  the  better 
empirical  taxonomic  studies. 

USAF  3306  Test  and  Evaluation  Squadron.  Mission  Handbook.  Edwards  Air  Force  Base, 
CA:  Author,  June  1977. 


The  3306  Test  and  Evaluation  Squadron  (TES)  is  responsible  for  instructional  systems 
development  for  the  Air  Force.  TES  has  developed  several  forms  to  assist  them  in  the  ISD 


process.  One  form  is  used  to  document  and  establish  training  requirements.  If  a  valid 
training  requirement  is  identified,  required  behaviors,  conditions,  and  standards  of 
performance  are  specified.  Media  requirements  are  also  detailed  in  this  statement. 
Another  form  is  used  in  detailing  maintenance  instructions  procedures  when  adequate  data 
are  not  available.  Constraining  factors,  resource  requirements,  and  performance  criteria 
are  also  defined. 

Wheaton,  G.  R.,  Rose,  A.  M.,  <5c  Fingerman,  P.  W.  Methods  for  predicting  job  abilit 
requirements:  IV.  Task  characteristics,  ability  requirements,  and  problem-solvin 


strategies.  Washington,  DC:  American  Institutes  for  Research,  September  1975. 

This  report  describes  four  studies  designed  to  generate  principles  for  identifying 
ability  requirements  from  the  task  characteristics  and  the  conditions.  Most  analyses  of 
the  abilities  necessary  to  perform  a  task  are  based  on  task  characteristics  and  not  on  the 
conditions  under  which  the  task  is  performed. 


Willis,  M.  P.,  &  Peterson,  R.  O.  Deriving  training  device  implications  from  learnin 
rinciDles.  Volume  I:  Guidelines  for  training  device  design.  develoDment  and  use. 


Port  Washington,  NY:  Naval  Training  Device  Center,  1961.  (AD-264  364) 


Basic  principles  of  skill  acquisition  and  a  detailed  behavioral  taxonomy  were  derived. 
These  two  dimensions  were  then  related  in  a  matrix  by  this  approach.  A  task  is  analyzed 
into  a  task  description,  list  of  critical  activities,  and  the  behavioral  categories  required. 
According  to  the  composition  of  skill  and  behavioral  categories  identified  for  a  given  task, 
particular  guidelines  for  training  are  identified.  Because  the  categories  are  rather 
subjective,  this  approach  is  relatively  difficult  to  apply. 

Media  Selection  Methods 

Anderson,  R.  H.  Selection  of  media:  Another  perspective.  Improving  Human  Perfor¬ 
mance:  A  Research  Quarterly,  1974,  3,  81-107. 


The  selection  of  instructional  media  is  difficult,  because  it  must  be  based  on  a 
combination  of  complex  and  interrelated  factors  such  as  degree  of  stimulus  and/or 
response  simulation  necessary,  hardware  availability,  cost-effectiveness,  and  need  for 
media  within  the  larger  context  of  an  overall  training  system.  As  the  decision  process 
now  stands,  there  are  very  few  guidelines  available.  Historical  data  on  behavioral  change 
through  media  use  are  imprecise.  Cost  figures  are  difficult  to  determine.  Training 
requirements  often  specify  the  need  for  a  particular  medium  within  the  requirements 
statement  itself.  A  series  of  key  decisions  is  suggested.  Is  one  interested  in  information 
or  instruction?  Next,  one  must  choose  between  instructional  aids  (media  to  aid  live 
instructor  presentation)  and  instructional  media  excluding  live  instructor.  The  third 
decision  involves  the  characterization  of  content  and  distribution  requirements.  Finally,  a 
particular  category  of  media  is  selected.  Flowcharts  are  developed  to  assist  the  user  in 
decision  making.  Most  of  the  media  considered  may  be  classified  as  communication 
media. 

Boeing  Company.  Specification  for  the  design,  preparation,  and  submission  of  training  and 
training  equipment  requirements.  Seattle:  Author,  1975.  ' 

Training  media  requirements  should  be  selected  to  facilitate  the  specified  learning 
types  required  to  achieve  behavioral  objectives.  The  intent  should  not  be  to  establish 
particular  types  of  media,  rather  to  identify  media  characteristics  that  training  special¬ 
ists  can  use  as  requirements  in  designing  media  and  equipment.  Media  characteristics  are 
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presented  in  five  classes:  visual/static,  visual/static/aural,  visual/dynamic,  visual/dy¬ 
namic/aural,  and  aural  only.  From  these  categories,  more  specific  media  alternatives 
may  be  identified.  Boeing  has  prepared  a  handbook  for  the  design  of  training  and  training 
equipment  for  Navy  airborne  weapon  systems  that  presents  a  more  detailed  discussion  of 
the  topic. 

Boucher,  G.  B.,  Gottlieb,  M.  3.,  <5c  Morganlander,  M.  L.  A  selection  system  and  catalog  for 
instructional  media  and  devices.  Bethpage,  NY:  Grumann  Aerospace  Corporation, 
October  1971. 

This  report  was  developed  for  media  selection  decisons  for  the  F-14  training  system. 
An  extensive  computer  data  bank  of  media,  media  characteristics,  and  associated  cost 
considerations  was  accumulated.  The  capability  exists  to  list  all  media  that  satisfy  a 
particular  set  of  characteristics.  A  media-capabilites  matrix  of  media  types  by  media 
characteristics  is  the  basis  for  the  selection  system.  Each  media-by-characteristic  cell 
may  be  assigned  one  of  a  set  of  values  depending  on  the  degree  to  which  that  medium 
contains  that  characteristic.  The  process  is  carried  out  by  first  identifying  the  desired 
characteristics,  and  then  determining  which  medium  optimizes  the  fit.  Data  processing 
may  be  automated  or  manual. 

Braby,  R.,  Henry,  3.  M.,  Parrish,  W.  F.,  3r.,  <5c  Swope,  W.  M.  A  technique  for  choosing 
cost-effective  instructional  delivery  system  (TAEG  Rep.  No.  16).  Orlando:  Navy 
Training  Analysis  and  Evaluation  Group,  April  1975. 

This  media  selection  approch  involves  three  major  steps:  (1)  redefining  training 
objectives  into  appropriate  learning  algorithms,  (2)  choosing  a  set  of  instructional  media 
that  meet  criteria  for  stimulus  demands,  training  setting  requirements,  and  administra¬ 
tive/budgetary  constraints,  and  (3)  applying  cost  estimates  to  each  alternative  media. 
The  relative  costs  of  alternatives  is  a  major  factor  entering  into  the  final  decision  to 
adopt  a  given  media/media  mix.  All  other  things  being  equal,  the  least  costly  media  are 
adopted  for  use. 

The  first  step  in  applying  the  media  selection  method  is  to  classify- -and,  thereby, 
match--each  training  objective  into  the  appropriate  one  of  12  learning  algorithms.  In  the 
next  step  the  user  consults  one  of  12  matrices  appropriate  to  the  algorithms  selected  for 
use  in  the  training  program.  The  rows  of  the  matrix  correspond  to  criteria  required  of 
media  in  order  to  successfully  apply  the  instructional  algorithm.  The  user  selects  from 
this  extended  list  those  criteria  required  for  his  particular  situation.  Columns  of  the 
matrix  correspond  to  the  different  media  that  can  appropriately  be  used  in  the 
instructional  algorithm.  The  user  selects  only  the  media  meeting  all  of  the  criteria  for 
selection  from  this  pool.  Typically,  two  or  more  alternative  media  are  selected  in  this 
way.  The  user  then  examines  this  reduced  set  of  alternative  media  closely  on  the  basis  of 
11  practical  criteria  such  as  budget  cycle  constraints,  learning  style  of  trainers,  time  to 
produce  the  system,  etc.  Based  on  this  analyses  additional  media  are  eliminated. 

Two  things  are  worth  mentioning  about  this  part  of  the  media  selection  procedure. 
(1)  The  range  of  media  under  consideration  is  extensive--from  classroom  lecture  through 
simulators.  This  allows  for  systematic  consideration  of  all  alternatives.  Selection  of 
training  devices  is  a  possible  output  of  this  seleciton  procedure.  Many  media  selection 
procedures  do  not  offer  this  alternative.  (2)  At  this  point  in  the  process,  formal  detailed 
costing  has  not  been  performed;  however,  general  cost  factors  are  considered.  The  media 
selection  charts  contain  a  cost-selective  decision  point  under  administrative  criteria. 
Also,  cost  related  issues  are  raised  in  the  test  of  practicality.  These  cost  considerations 
can  eliminate  high-cost  media  which  many  media  selection  procedures  cannot  do. 
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The  final  step  is  the  formal  economic  analysis  of  the  remaining  reduced  set  of  media, 
which  involves  detailed  cost  estimates  of  facilities,  equipment,  instructional  material 
development,  personnel,  supplies,  and  students.  Application  of  a  cost  model  in  the 
analysis  produces  a  present  cost  figure  of  each  alternative  media.  The  model  is  not 
intended  to  forecast  total  system  costs,  but  to  aid  in  media  selection.  The  model  does  not 
provide  a  cost-benefit  analysis. 

Bretz,  R.  The  selection  of  appropriate  communication  media  for  instruction:  A  guide  for 
designers  of  Air  Force  technical  training  programs  (Tech.  Rep.  R-6Q1-PR).  Santa 
Monica:  Rand  Corporation,  February  1971. 

Eight  classes  of  communication  media  are  identified:  audio-motion-visual,  audio¬ 
still-visual,  audio-semimotion,  motion-visual,  semimotion,  still-visual,  audio,  and  print.  It 
is  important  to  note  that  all  eight  alternative  media  are  telecommunications  or  recorded 
media.  Each  category  of  media  is  illustrated  by  relevant  examples.  A  series  of  flow 
charts  is  provided  to  assist  the  user  in  selecting  most  appropriate  medium  for  a  given 
situation.  By  answering  yes  or  no  at  each  decision  point,  the  user  is  directed  to  the 
appropriate  media  class. 

Briggs,  L.  3.  Handbook  of  procedures  for  the  design  of  instruction.  Pittsburgh:  American 
Institutes  for  Research,  1970. 

The  Briggs  handbook  is  oriented  to  the  teaching  of  academic  subjects  in  the 
classroom.  Seventeen  conditions  of  learning  are  identified  and  related  to  Gagne's  eight 
types  of  learning.  The  14-step  procedure  presented  is  highly  dependent  on  a  user's  own 
relevant  experience.  Specific  media  categories  are  not  specified.  A  designer  is  asked  to 
choose  media  that  meet  criteria  established  by  the  conditions  of  learning  and  related 
stimulus  characteristics. 

Campbell,  3.  O.,  &  Hughes,  J.  Optimizing  media  selection.  Proceedings  of  the  11th 
NTEC/Industrial  Conference,  Orlando,  November,  1978. 

A  procedure  for  selecting  the  optimum  media  for  study  device  session  is  presented. 
Media  selection  involves  three  components:  (1)  determination  of  media  requirements  for 
each  objective,  (2)  analysis  of  the  requirements  for  individual  objectives  into  optimum 
mixes  of  training  media,  and  (3)  selection  of  a  final  media  mix  based  on  cost  and 
availability.  This  procedure  accomplishes  the  first  two  components.  The  1 1  steps  in  the 
procedure  are  described  in  detail.  The  result  of  this  procedural  approach  is  that  all 
trainers,  except  the  last  of  each  type,  will  have  enough  objectives  assigned  to  them  to  fill 
all  available  time.  Following  this  process,  it  is  necessary  to  develop  a  detailed  set  of 
functional  specifications  and  detailed  costing  and  support-requirements  for  each  proposed 
trainer.  This  must  be  done  with  the  manufacturer  of  the  equipment. 

Carpenter,  P.  An  overview  of  MODIA:  A  method  of  designing  instructional  alternatives 
for  Air  Force  training.  Santa  Monica:  Rand  Corporation,  November  1972.  (AD-760 
04 6) 

Rand  Corporation  is  developing  a  model  to  augment  the  Air  Force's  Instructional 
Systems  Development  model.  This  model- -referred  to  as  MODIA,  Method  of  Designing 
Instructional  Alternatives- -addresses  administrative  and  training  aspects  of  training 
program  design.  Media  taxonomies  and  cost  effectiveness  of  media  categories  are 
examined.  MODIA  is  computerized  and  designed  to  aid  the  decision  maker  in  exercising 
his  responsibilities. 
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Department  of  the  Air  Force.  Instructional  system  development  (AF  Manual  50-2). 
Washington,  DC:  Author,  July  1975. 

This  technique  is  based  on  a  matrix  consisting  of  six  types  of  learning  objectives  and 
ten  types  of  instructional  media.  Each  matrix  cell  receives  a  low,  medium,  or  high 
classification  that  provides  an  indication  of  the  usefulness  of  that  medium  for  achieving 
the  particular  learning  objective. 

Department  of  the  Navy.  Military  specification:  Training  requirements  for  aviation 
weapon  systems,  MIL-T-29053.  Washington,  DC:  Author,  October  1977. 

This  Navy  specification  contains  a  detailed  media  selection  model  presented  in  terms 
of  relevant  questions,  alternative  paths  based  on  answers  to  the  questions,  and  ultimately 
recommendations  for  appropriate  media.  Important  variables  affecting  the  media 
selection  process  include  instructional  display  requirements,  student  response  detection 
capabilities,  feedback  requirements,  and  student  program  interaction  level.  Student 
program  interaction  covers  hands-on  objectives,  information  storage  and  generation 
requirements,  and  program  constraints  such  as  cost  and  safety.  The  media  alternatives  to 
be  considered  must  represent  the  range  of  candidate  media  falling  within  the  constraints 
of  program  feasibility. 

The  representative  media  types  considered  in  this  model  include  mediated  interactive 
lectures,  workbooks,  slide-tape  presentations,  random  access  slide  presentations,  video¬ 
tape  presentations  (animated  videotape,  level-three  videotape,  level-two  videotape,  level- 
one  videotape,  home-movie  level  videotape),  computer-assisted  instruction,  and  hands-on 
exercises.  Instructional  requirements  for  training  devices  are  described  in  terms  of 
functional  purpose  and  required  capability,  known  or  estimated  training  equipment 
hardware  and  software  functional  capabilities,  distinguishing  capabilities  from  alternative 
instructional  features,  and  criticality  to  training.  This  procedure  appears  to  offer  much 
promise. 

Eckstrand,  G.  A.  Current  status  of  the  technology  of  training  (Tech.  Rep.  64086). 
W right-Pa tterson  Air  Force  Base,  OH:  September  1964. 

This  report  presents  an  overview  of  training  technology.  The  processes  of  determin¬ 
ing  training  requirements,  developing  training  environments,  and  measuring  training 
results  are  reviewed.  Media  decisions  occur  during  the  development  of  an  appropriate 
training  environment.  The  complexity  of  military  jobs  and  requirements  for  training 
efficiency  have  led  to  the  increased  role  of  training  devices  in  training  systems 
development.  However,  engineering  technology  related  to  simulation  has  grown  more 
rapidly  than  our  ability  to  specify  the  characteristics  that  a  particular  simulator  should 
have  to  most  effectively  accomplish  specified  training  objectives.  Therefore,  physical 
fidelity  is  often  emphasized  at  the  expense  of  functional  fidelity.  Information  exists  on 
each  type  of  media,  however,  procedures  for  selecting  media  to  meet  specific  require¬ 
ments  is  still  in  a  preliminary  stage  of  development. 

Johnson,  S.  L.,  Buchenmaier,  C.  C.,  Sc  Sugarman,  R.  C.  Instructional  system  design  for 
aircrew  training.  Proceedings  of  the  Eighteenth  Annual  Meeting  of  the  Human  Factors 
Society,  Huntsville,  AL,  October  1974. 

This  paper  discusses  the  methodology  and  output  of  the  instructional  systems 
development  (1SD)  process  for  the  crewmembers  of  the  B-l.  Selection  of  appropriate 
media  for  the  academic  portion  of  the  training  program  has  a  large  impact  on  the  time¬ 
phasing  of  the  instructional  system.  Advances  in  computer-assisted  and  managed 


instruction  for  pilot  training  are  relevant  to  media  decisions.  Relative  advantages  of 
individualized  instruction  and  integrated  crew  training  can  be  assessed.  The  hardware 
technology  available  for  simulation  in  training  is  ahead  of  the  behavioral  technology 
availabe  for  making  adequate  media  decisions. 

Jorgensen,  C.  C.  A  methodology  and  analysis  for  cost-effective  training  in  the  AN/TSQ- 
73  missile  minder.  Fort  Bliss,  TX:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  September  1976. 

Jorgensen  proposes  media  selection  by  performing  a  cost  and  training  effectiveness 
analysis  (CTEA)  within  a  cost  and  operational  effectiveness  analysis  (COEA).  Jorgensen 
formed  a  matrix  with  55  generic  media  characteristics  on  one  axis  and  75  media  types  on 
the  other  axis.  Next,  judges  assigned  applicability  ratings  to  each  matrix  cell  based  on 
the  degree  to  which  each  type  of  media  delivers  each  of  the  characteristics.  Then,  tasks 
to  be  trained  were  assigned  to  the  various  generic  media  characteristics  appropriate  to 
their  use  in  training.  Those  media  that  contained  the  largest  number  of  applicable 
characteristics  for  the  tasks  were  then  selected  for  a  detailed  cost  analysis  according  to 
the  computerized  cost  model. 

Kinkade,  R.  G.,  &  Wheaton,  G.  R.  Training  device  design.  In  Van  Cott  and  Kinkade  (Eds.), 
Human  Engineering  Guide  to  Equipment  Design.  Washington,  DC:  U.S.  Government 
Printing  Office,  1972,  667-699. 

The  authors  qualify  levels  of  training  into  indoctrination,  procedural,  familiarization, 
skill,  and  transition.  They  associate  different  types  of  training  devices  with  each  level. 
Training  devices  are  then  divided  into  training  aids  (e.g.,  films,  TV)  and  training 
equipment.  The  training  equipment  category  includes  part- task  trainers,  such  as  driver 
trainers,  and  whole  task  trainers,  such  as  weapon  system  simulators.  At  higher  level  types 
of  training,  training  equipment  is  generally  recommended.  Training  aids  appear  to  be 
recommended  primarily  for  passive  learning  tasks. 

Narva,  M.  A.  Development  of  a  systematic  methodology  for  the  application  of  judgmental 
data  to  the  assessment  of  training  device  concepts.  Paper  presented  at  the  U.S.  Army 
Science  Conference,  West  Point,  June  1978. 

The  paper  considers  the  time  constraints  resulting  from  concurrent  development  of 
prime  and  training  systems.  A  method  called  TRAINVICE  was  developed  to  assist  in  the 
development  of  training  concepts.  Task  commonality,  physical  characteristics,  functional 
characteristics,  skill  and  knowledge  requirements,  and  task  training-difficulty  ratings  are 
all  derived  as  part  of  the  training  effectiveness  analysis.  These  ratings  integrate 
judgmental  processes  within  a  framework  that  can  be  used  during  the  development  of 
training  devices. 

Nawrocki,  L.  Paper  presented  at  Computer  Systems  Tutorial  at  the  22nd  Annual  Meeting 
of  the  Human  Factors  Society.  Detroit,  October  1978. 

The  future  of  videodiscs  as  media  for  instructional  delivery  is  discussed.  Videodiscs 
can  store  much  more  information  than  magnetic  discs.  The  outstanding  capabilities  of 
videodisc  technology  with  educational  implications  are:  (1)  automatic  frame  stop,  (2)  dual 
audio  track,  (3)  random  access  of  visual  scenes  as  opposed  to  serial  access  of  other 
available  media  (e.g.,  videotape,  film),  and  (4)  fast  forward  or  reverse  search. 


A-27 


Clearly,  the  impact  of  videodisc  will  be  substantial  in  the  5-  to  10-year  planning 
horizon.  Any  weapon  system  originating  in  the  next  few  years  should  consider  using  this 
technological  advancement. 

Nunnelly,  C.  L.,  Klemmer,  A.  G.,  Corrigan,  R.  E.,  <5c  Kaufman,  R.  A.  The  instructional 
system  approach  to  maintenance  technical  training:  Development  and  implimentation 
model.  Human  Factors,  1966,  8,  163-172. 

This  article  presents  an  instructional  system  model  for  meeting  the  maintenance 
training  requirements  for  weapon  systems.  The  model  proceeds  in  five  phases:  system 
analysis,  equipment/hardware  analysis,  human  component  analysis,  synthesis  of  hardware 
and  software  into  precise  training  requirements,  and  method/media  selection.  Methods 
refer  to  the  strategies  for  achieving  desired  terminal  performance,  while  media  are  the 
tools  by  which  the  methods  are  implemented.  The  stages  of  this  phase  include  task 
identification,  response  characterization,  simulation  criticality  and  learning  difficulty, 
learning  method,  and  instructional  setting.  The  emphasis  in  this  article  is  on  organizing 
decisions,  rather  than  replacing  subjective  inputs  of  analysts. 

Parker,  3.  F.,  Jr.,  &  Downs,  J.  E.  Selection  of  training  media  (Tech.  Rep.  ADS  TR  61- 
473).  Wright-Patterson  Air  Force  Base,  OH:  Aeronautical  Systems  Division,  September 
1961. 

Parker  and  Downs  discussed  the  translation  of  statements  of  desired  personnel 
performances  and  capabilities  and  task  analysis  documents  into  training  objectives.  They 
identify  three  classes  of  simulation  of  the  contextual  environment.  Extraneous  contextual 
cues  include  general  irrelevant  environment  cues  accompanying  task  performance.  Per¬ 
formance  degraders  are  forces  that  cannot  be  overcome  to  any  appreciable  extent  by 
practice  and  tend  to  lessen  performance  capabilities  in  a  relatively  constant  manner. 
Performance  disruptors  are  forces  that  are  disruptive  at  first,  but  can  be  overcome 
through  practice.  Therefore,  performance  disruptors  should  be  simulated,  while  there  is 
little  benefit  to  simulating  performance  degraders. 

Rhode,  W.  E.,  Esseff,  P.  J.,  Pusin,  C.  J.,  Quirk,  F.  B.,  &  Shulik,  R.  Analysis  and  approach 
to  the  development  of  an  advanced  multi-media  instructional  system.  Vol.  I  (Tech.  Rep. 
69-30).  Wright-Patterson  Air  Force  Base,  OH:  Air  Force  Human  Resources  Laboratory, 
Air  Force  Systems  Command,  May  1970. 

Westinghouse  Learning  Corporation  conducted  a  study  to  review  and  assess  current 
instructional  media  in  terms  of  functional  description,  instructional  flexibility,  support 
requirements,  and  costs.  A  model  of  an  individual  instructional  system  is  suggested. 
Requirements  and  features  of  automated  management  information  and  control  subsystems 
are  outlined,  and  major  features  for  developing  such  a  system  are  described.  This  study 
provides  a  needed  data  base  for  future  planning.  The  instructional  media  were  portable 
instructor  aids,  television,  portable  VTR  systems,  student  response  systems,  dial  access 
informaton  retrieval  systems,  learning  laboratories,  programmed  text,  teaching  machines, 
learner  centered  audiovisual  devices,  simulators,  and  advanced  concepts.  The  report  was 
designed  for  use  with  a  broad  range  of  training  tasks,  and  not  intended  to  be  a  media 
selection  technique.  However,  an  abundance  of  valuable  data  to  support  a  training  system 
developer  in  media  related  decisions  is  provided. 
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Siegel,  A.,  <5c  Federman,  P.  3.  Development  of  a  method  for  deriving  required  training 
aids/devices  and  applications  to  the  tactical  coordinator  position  in  ASW  aircraft  (Tech. 
Rep.  NAVTRADEVCEN  68-C-0212-1).  Orlando:  Naval  Training  Device  Center,  June 
1970. 

Two  scales  were  developed  to  rate  the  amount  and  taxation  level  of  intellectual 
activity  required  in  performing  a  task.  The  intellectual  operations  are  cognition,  memory, 
divergent  production,  convergent  production,  and  evaluation.  These  intellectual  opera¬ 
tions  are  based  on  Guilford's  structure  of  intellect  model.  Values  for  the  amount  of 
intellectual  activity  and  taxation  level  of  intellectual  activity  are  combined  with 
predetermined  values  of  the  suitability  for  each  instructional  medium  to  exercise  each  of 
the  intellectual  operations.  These  values  are  obtained  by  averaging  judgments  of  a  group 
of  subject  matter  experts.  Resulting  numerical  values  are  used  to  determine  if  a  training 
objective  is  sufficiently  difficult  or  demanding  to  require  media  support.  If  the  threshold 
value  is  attained  or  passed,  these  values  are  then  used  to  establish  a  ranking  of 
appropriate  selected  media.  The  media  include  programmed  instruction,  slides,  gaming, 
dynamic  trainers,  television,  and  eleven  additional  methods. 

Spangenberg,  R.  W.,  Riback,  Y.,  in  Moon,  H.  L.  The  state  of  knowledge  pertaining  to  cost- 
effective  training  methods  and  media  (HumRRO  Tech.  Rep.  73-13).  Alexandria: 
Human  Resources  Research  Corporation,  1973. 

This  report  includes  a  reviw  and  analysis  of  relevant  media-selection  procedures  for 
cost  effective  use  of  media  in  Army  training  programs.  Existing  training-methods  and 
media-selection  procedures  and  training  cost  analyses  are  not  adequate.  The  inadequacies 
relate  to  two  basic  problems:  procedures  developed  for  civilian  use  do  not  apply  to  Army 
conditions  and  existing  procedures  are  too  general  for  specific  application. 

To  create  the  external  conditions  required  for  learning,  media  must:  present  stimuli, 
direct  student  activities,  provide  models  for  terminal  performance,  provide  prompts, 
guide  thinking,  induce  transfer  of  knowledge,  and  provide  feedback.  Media-selection 
approaches  must  include  exhaustive  lists  of  alternative  types,  media  definitions  must  be 
based  on  media  attributes,  and  the  definitions  must  be  relatable  to  user  identified  media 
requirements.  Defining  media  in  terms  of  aggregate  sets  of  capabilities  appears  to  be  a 
reasonable  method  for  objective  definition. 

Sugarman,  R.  C.,  Buchenmaier,  C.  C.,  &  Johnson,  S.  L.  Systems  approach  to  training 
(SAT)  for  the  B-l  aircrew.  Proceedings  of  the  Eighteenth  Annual  Meeting  of  the  Human 
Factors  Society,  Huntsville,  AL,  October  1974. 


Development  of  a  methods/media  catalog  to  serve  as  an  aid  in  the  selection  of 
presentation  techniques  is  suggested.  The  data  in  the  catalog  would  serve  to  analyze 
simulation  technology.  Factors  of  cost,  availability,  support  personnel  and  material 
requirements,  and  facilities  requirements  would  be  specified.  Behavioral  technology 
advances  in  transfer  of  training  and  learning  theory  research  can  be  introduced  in  terms 
of  their  implications  for  teaching  methods  and  training  equipment. 

USAF  3306  Test  and  Evaluation  Squadron.  Mission  Handbook.  Edward  Air  Force  Base, 
CA:  Author,  June  1977. 

This  handbook  describes  in  detail  the  Air  Force  3306  Test  and  Evaluation  Squadron's 
approach  to  instructional  systems  development  (ISD).  The  handbook  presents  an  ISD 
model  for  new  weapon  systems.  A  section  in  the  handbook  addresses  the  selection  of 
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training  media.  Subject  matter  experts  (SMEs)  review  each  behavioral  requirement  and 
determine  the  best  media  to  satisfy  it.  A  rationale  checklist  and  training  media 
analogram  are  used  by  the  SME  to  reach  media  selection  decisions  more  objectively. 
Instructional  media  are  reviewed  as  supplemental  (i.e.,  media  are  designed  to  accomplish 
a  training  objective  that  cannot  be  accomplished  by  an  instructor  alone).  Not  all  training 
objectives  require  the  use  of  media.  A  real  need  must  be  demonstrated  before  media  are 
procured.  The  media  analogram  is  a  valuable  aid  in  selecting  a  particular  class  of  media. 

Walker,  R.  W.  An  evaluation  of  training  methods  and  their  characteristics.  Human 
Factors,  1965,  7,  347-354. 

When  technical  training  specialists  and  supervisors  were  asked  to  list  the  factors  they 
used  to  select  training  techniques,  34  factors  were  identified.  A  group  of  training 
specialists  was  then  asked  to  rate  each  of  16  training  techniques  on  each  of  the  selection 
factors.  Based  on  these  data,  a  matrix  was  produced  that  contained  average  values  for 
each  training  technique  by  selection  factor.  Training  program  designers  may  use  the 
matrix  by  specifying  real-world  conditions  such  as  budgetary  restrictions,  personnel 
considerations,  and  equipment  requirements.  They  use  the  matrix  to  select  the  training 
technique  best  suited  to  the  given  operational  parameter  set. 

Wheaton,  G.  R.,  Fingerman,  P.  W.,  Rose,  A.  M.,  <5c  Leonard,  R.  L.,  Jr.  Evaluation  of  the 
effectiveness  of  training  devices:  Elaboration  and  application  of  the  predictive  model 
(Res.  Memo.  76-16).  Alexandria:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  1976. 

Planning,  designing,  and  developing  a  training  program  incorporating  required  devices  is 
difficult.  One  begins  by  stating  training  requirements,  identifying  what  needs  to  be 
trained,  and  then  specifying  a  general  training  system  to  meet  these  objectives.  At  this 
stage,  the  specification  is  functional,  rather  than  hardware  oriented.  Converting  the 
functional  specification  into  hardware  is  done  systematically.  The  TRAINVICE  model 
presents  an  approach,  based  on  a  transfer  of  training  model,  to  predict  and  evaluate  the 
effectiveness  of  training  devices.  The  model  is  presently  not  used  in  selecting  media,  but 
the  learning  theory  approach  may  have  application  to  media  selections. 

Training  Program  Assessment 

Branch,  K.  V.,  Duffy,  L.  R.,  &  Staley,  J.  D.  Design  of  training  systems,  phase  IV  report 
(TAEG  Tech.  Rep.  37).  Orlando:  Training  Analysis  and  Evaluation  Group,  October 
1976.  (AD-A046  700) 


Branch,  K.  V.,  Staley,  J.  D.,  <5c  Yanko,  R.  E.  Design  of  training  systems,  users  manual 
(TAEG  Rep.  36).  Orlando:  Training  Analysis  and  Evaluation  Group,  September  1976. 
(AD-A040  238) 

Duffy,  L.  R.  Design  of  training  systems  utility  assessment  (Tech.  Rep.  TAEG-33). 
Orlando:  Training  Analysis  and  Evaluation  Group,  May  1976.  (AD-A026  289) 

Duffy  (1976);  Branch,  Duffy,  and  Staley  (1976);  and  Branch,  Staley,  and  Yanko  (1976) 
all  discussed  the  Design  of  Training  systems  (DOTS)  project  in  which  two  computer  based 
models  were  designed,  developed,  and  validated.  The  first  model--the  system  capabil¬ 
ities/requirements  and  resources  (SCRR)  model — incorporates  operations  research  tech- 
niqes  to  optimize  training  student  throughput  and  resource  utilization,  given  an  extensive 
data  base  describing  resources  available  and  the  required  student  throughput.  The 
training  process  flow  (TPF)  model  has  as  its  input,  the  output  of  the  SCRR  model  and 
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predicts  training  system  performance  based  on  statistical  analyses  of  historical  data  and 
computer  simulation.  The  overall  package  was  intended  to  permit  rapid  determination  of 
the  effects  of  potential  training  system  alterations  (e.g.,  resource  availabilities  and 
assumptions).  Additionally,  it  was  designed  to  be  used  in  an  interactive  mode,  if  desired, 
to  permit  an  iterative  exploration  of  alternatives. 

Goldstein,  I.  L.  A  systems  approach  to  studying  organizations:  Selection,  training, 
performance.  Proceedings  of  the  Human  Factors  Society  22nd  Annual  Meeting,  Detroit, 
October  1978,  88-90. 

In  assessing  effectiveness  of  training  programs,  the  usual  approach  is  to  measure 
performance  before  and  after  instruction  and  make  comparisons  to  appropriate  controls. 
While  this  approach  provides  valuable  information,  the  author  feels  it  ignores  many 
aspects  of  the  training  process.  Data  obtained  during  training  may  be  used  to  predict  on- 
the-job  performance.  The  potential  for  enhanced  predictability  may  be  cost  effective 
relative  to  the  operational  system  in  some  circumstances.  Where  cost  of  training  is  an 
important  factor,  one  can  use  other  measures  to  predict  training  performance  and,  then, 
use  actual  training  performance  to  predict  job  performance.  Another  area  in  which 
training  programs  can  influence  job  performance  is  the  training  expectations  and  job 
expectations  that  the  trainees  bring  with  them.  Realistic  job  previews  during  recruitment 
have  been  found  to  reduce  the  number  of  early  turnovers  within  an  organization.  Since 
the  training  program  is  the  point  of  entry  into  an  organization,  realistic,  and  accurate 
information  during  training  might  smooth  the  transition  from  training  to  job,  which  might 
improve  the  chances  of  job  success. 

Grace,  G.  L.  Human  effectiveness  and  evaluation.  Proceedings  of  the  Human  Factor 
Society  18th  Annual  Meeting,  Huntsville,  AL,  October  1974,  57-60. 

Recent  social  and  economic  developments  in  this  country  have  increased  the 
importance  of  the  human  factors  of  effectiveness,  that  is  the  motivation  and  utilization 
of  human  resources.  The  process  of  evaluation  is  one  method  of  determining  improvement 
in  human  effectiveness.  This  paper  describes  the  evaluation  of  two  programs  (one  of 
which  is  the  Navy  Career  Counseling  Program)  designed  to  increase  effectiveness.  As  a 
result  of  the  evaluation,  many  modifications  have  been  implemented  in  this  program. 

Hall,  E.  R.,  Lam,  K.,  <5c  Bellamy,  S.  G.  Training  effectiveness  assessment:  Volume  I, 
current  military  training  evaluation  programs  (TAEG  Rep.  No.  39).  Orlando:  Training 
Analysis  and  Evaluation  Group,  December  1976. 

Philosophy,  methods  of  documentation,  and  current  practices  in  training  effective¬ 
ness  assessment  are  reviewed  for  Air  Force,  Navy,  Army,  and  Marine  Corps  programs. 
Each  program  is  evaluated  in  terms  of  apparent  strengths  and  weaknesses.  Considerable 
variability  exists  in  the  way  each  of  the  services  defines  and  implements  it  training 
evaluation  activities.  Some  of  the  variability  is  dt  e  to  differences  in  mission.  For 
example,  the  Army  concept  is  keyed  to  contributions  to  combat  readiness,  as  opposed  to 
the  effectiveness  of  particular  courses. 

Hall,  E.  R.,  Rankin,  W.  C.,  <5c  Aagard,  J.  A.  Training  effectiveness  assessment:  Volume  11, 
problems,  concepts,  and  evaluation  alternatives  (TAEG  Rep.  No.  39).  Orlando:  Training 
Analysis  and  Evaluation  Group,  December  1976. 


Specific  problems  affecting  training  evaluation  programs  are  identified.  General 
procedures  are  presented  for  assessing  the  effectiveness  of  training  courses  for  gathering 
required  evaluation  data.  One  recommendation  is  for  the  establishment  of  a  training 


effectiveness  assessment  center  to  serve  as  a  centralized  agency  assisting  in  the  planning 
and  conduct  of  training  evaluations. 


A  number  of  factors  can  limit  the  value  of  training  evaluation  programs.  These  are 
intensified  by  some  negative  attitudes  toward  evaluation  in  general,  and  these  attitudes 
can  be  made  worse  as  a  result  of  administrative  problems  and  lack  of  command  emphasis. 
Greater  emphasis  is  also  needed  to  train  military  personnel  to  conduct  detailed  training 
analyses.  Unless  these  factors  are  systematically  addressed,  training  evaluation  programs 
will  continue  to  suffer. 

Holshouser,  E.  L.  Human  factors  engineering  test  and  evaluation  technology.  Proceedings 
of  the  Human  Factor  Society  18th  Annual  Meeting,  Huntsville,  AL,  October  1974,  343- 
349. 

In  implementing  Department  of  Defense  Directive  5000.1  (outlining  policy  for  major 
defense  system  acquisition),  the  Naval  Air  System  Command  has  proposed  human  factor 
testing  and  evaluation  (T&E)  of  any  system  requiring  the  approval  and  processing  of  the 
Defense  Systems  Acquisitions  Review  Council  (DSARC).  The  framework  of  this  new 
policy  is  a  series  of  decision  points  addressing  program  need,  cost,  and  direction.  At  each 
decision  point,  critical  questions  and  issues  are  resolved  through  T&E  before  moving  to 
the  next  decision  point.  The  ultimate  goal  is  to  ascertain  operational  suitability  of  new 
Navy  systems  through  systematic  methodologies  of  T&E  technology.  The  proposed  system 
provides  a  clearing  house  for  documentation  storage  and  retrieval  of  human  factors  T&E 
data  for  use  by  the  developers  of  future  systems. 

Jensen,  H.  E.,  &  Valentine,  L.  D.,  Jr.  Validation  of  ASVAB-2  against  civilian  vocation¬ 
al/technical  high  school  criteria  (AFHRL  TR-76-16^  Lackland  Air  Force  Base,  TX: 
Personnel  Research  Division,  Air  Force  Human  Resources  Laboratory,  March  1976. 
(AD-A023  118) 

The  recently  developed  Armed  Services  Vocational  Aptitude  Battery  (ASVAB-2)  was 
taken  by  a  high  school  population  and  the  results  compared  against  later  performance  by 
the  population  in  civilian  vocational  technical  courses.  The  data  indicate  that  ASVAB's 
utility  is  equal  to  or  somewhat  superior  to  commercial  aptitude  batteries.  Statistical  data 
are  given  for  41  vocational-technical  fields. 

Leuba,  H.  R.  A  systems  approach  comes  to  personnel  management.  Proceedings  of  the 
Human  Factors  Society  19th  Annual  Meeting,  Dallas,  October  1975,  467-472. 

The  systems  approach  to  problem  solving  was  applied  to  the  problem  of  personnel 
performance  evaluation.  Data  were  collected  and  analyzed  to  define  the  purpose, 
enumerate  the  characteristics,  and  discuss  the  mechanisms  used  to  implement  a  system  of 
personnel  performance  evaluation.  A  system  must  have  a  well-defined  purpose  and  be 
based  on  the  list  of  80  characteristics  outlined.  The  mechanisms  to  be  used  depend  on  the 
needs  of  the  using  organization.  Any  operating  system  should  be  flexible  enough  to  allow 
employees  and  managers  to  be  involved  at  the  level  of  participation  in  the  system  meeting 
their  needs  at  the  time.  Additional  details  are  available  to  those  who  want  to  try  this 
solution. 

Lindahl,  W.  E.,  &  May,  D.  E.  Programming  Navy  specialized  training.  Report  prepared 
for  the  Deputy  Assistant  Secretary  of  the  Navy  (Manpower),  August  1978. 

Results  of  research  and  analysis  of  the  specialized  training  programming  process 
were  identified.  In  this  report,  procedures  specified  in  the  training  portion  of  the  Navy's 


Lockway,  M.  R.,  <5c  Yasutake,  3.  Y.  The  evolution  of  the  Air  Force  advanced  instructional 
system.  The  Second  Symposium  on  Cost  Effective  Learning  Through  the  Application  of 
Computer  Technology,  Washington,  DC,  February  1973,  112-133. 

This  paper  presents  an  overview  of  the  Air  Force's  Advanced  Instructional  System 
AIS),  which  is  a  proposed  computer-based  multimedia  system  for  the  administration  and 
nanagement  of  individualized  technical  training  on  a  large  scale.  The  AIS  system  is 
iroken  down  into  seven  subsystems:  (1)  instructional  materials,  (2)  instructional  strate- 
;ies,  (3)  media,  (4)  computer  software,  (5)  computer  hardware,  (6)  personnel  and  training, 
nd  (7)  related  subsystems. 

The  authors  contend  that  the  uniqueness  of  the  AIS  is  expressed  in  several  ways:  (1) 
t  is  the  first  attempt  to  integrate  all  of  the  technology  required  for  cost  effective 
mplementation  of  large-scale  individualized  instruction.  (2)  It  provides  the  capability  of 
ising  several  methods  and  media  rather  than  just  programmed  instruction  or  CAI.  (3)  It 
ias  capacity  for  incorporating  instructional  and  management  innovations  as  they  become 
vailable.  (4)  It  has  inherent  capacity  for  continuously  evaluating  and  upgrading  its  own 
:ost  effectiveness.  (5)  AIS  is  designed  as  a  total  system  to  perform  all  of  the 
nstructional  system  design,  administration,  and  management  required  to  conduct  a  large- 
cale  training  enterprise. 

ugarman,  R.  A  second  chance  for  computer-aided  instruction.  IEEE  Spectrum,  August 
1978. 

In  this  article,  the  shortcomings  of  CAI  in  the  last  decade  are  attributed  to  the  lack 
i  proper  courseware  development,  and  not  to  the  inadequacy  of  the  media  itself.  New 
ourse-generation  languages,  such  as  Tutor,  provide  the  opportunity  for  significant 
mprovements  in  the  near  future.  Sugarman  also  suggests  that  the  economics  of  going 
rom  large  time-sharing  computer  systems  to  support  CAI  to  low-cost  stand-alone  systems 
nay  encourage  more  institutions  to  consider  CAI  as  an  instructional  medium.  However, 
he  large  amount  of  preparation  time  per  hour  of  lecture  (estimated  at  50  hours  per  hour 
>f  lesson)  still  appears  to  be  a  major  stumbling  block  for  full-scale  acceptance  of  CAI  by 
he  education  and  training  community. 

ugarman,  R.  What's  new  teacher?  Ask  the  computer.  IEEE  Spectrum,  September  1978. 

The  following  applications  of  CAI  are  proposed:  (1)  It  can  permit  the  learner  to  see 
/ hat  he  is  doing  and  what  progress  has  been  made.  (2)  It  can  function  as  an  "articulate 
xpert,"  by  using  the  large  information  storage  capabilities.  (3)  It  can  be  used  as  a  home 
ibrary,  by  employing  videodisc  technology.  (4)  It  can  be  used  as  a  low  cost  simulator  to 
void  the  costs  of  expensive  simulator  hardware. 

'alverde,  H.,  &  Morgan,  R.  L.  Influence  on  student  achievement  of  redundancy  in  self- 
instructional  materials  (Tech.  Rep.  AMRL-TR-68-75).  W right-Pa tterson  Air  Force 
Base,  OH:  Aerospace  Medical  Research  Laboratory,  December  1968. 

This  study  investigates  the  effect  of  redundancy  on  learning  from  self-instructional 
laterial.  Redundancy  is  defined  as  the  use  of  more  words,  instructional  materials,  or 
upport  materials  than  are  necessary  to  teach  desired  behaviors.  The  results  indicated 
hat  eliminating  the  usual  redundancy  in  a  linear  program  significantly  increased  student 
chievement. 


McCann,  P.  H.,  Lahey,  G.  F.,  &  Hurlock,  R.  E.  A  comparison  of  student  option  versus 
program  controlled  CAI  training  (Tech.  Rep.  SRR  73-17)  San  Diego:  Naval  Personnel 
and  Training  Research  Laboratory,  April  1973. 

Students  were  allowed  to  select  their  training  from  an  index  of  descriptive  lesson 
Dbjectives  or  they  were  pretested  just  before  each  lesson  objective  and,  then,  branched  to 
the  training  sequence  appropriate  to  their  level.  A  pretraining  narrative  presentation  was 
given  to  half  the  students.  No  significant  differences  were  found  in  test  performance  or 
training  time  measure  in  any  of  the  four  experimental  conditions.  Students  who  selected 
their  own  training  had  more  favorable  attitudes  toward  CAI.  A  pretraining  narrative  was 
felt  to  be  a  valuable  aid. 

Pollack,  H.  Time  and  cost  effective  techniques  in  supportive  computerized  training 
systems.  Proceedings  of  the  6th  NTEC/Industry  Conference,  Orlando,  1973. 

The  author  concludes  that  computer  usage  will  remain  in  a  supportive  role  in  the 
educational  process  until  its  inability  to  make  adjustments  in  lesson  presentation  through 
student  feedback  is  accomplished.  Steep  hardware  and  telephone  transmission  costs  are  a 
large  impediment  toward  a  more  pervasive  usage  of  CAI  throughout  the  nation.  To 
maximize  its  cost  effectiveness,  the  computer  should  be  used  only  for  those  activities  in 
which  it  clearly  surpasses  all  other  media. 

Pollack  suggests  that:  (1)  Text  storage  and  graphics  should  be  left  to  workbooks  and 
handouts  used  to  minimize  storage  and  equipment  expense.  (2)  Randomization  of  data, 
calculations,  and  management  of  student  records  are  fortes  of  the  computer  and  will  save 
student  and  instructor  time.  (3)  Socratic  method  of  lesson  development  should  be  left  up 
to  the  human  teacher. 


Rigney,  J.  W.  A  discussion  of  behavioral  technology  laboratories  CAI  projects  in  relation 
to  a  CAI  test-bed  concept  (Tech.  Rep.  71).  Los  Angeles:  University  of  California, 
Department  of  Psychology,  July  1973. 

The  intent  of  the  report  is  to  present  the  major  elements  of  computer-assisted 
instruction  (CAI),  relate  them  to  laboratory  research  findings,  and  describe  projects 
underway  at  Behavioral  Technology  Laboratories.  CAI  elements  are  discussed  with 
reference  to  a  block  diagram  depicting  13  elements,  their  grouping,  and  interactions.  In  a 
sense,  this  CAI  model  represents  a  generalized  instructional  system.  Major  focus  of  the 
discussion  concerns  the  elements  that  are  collectively  called  internal  processing.  This 
block  represents  the  students'  internal  subjective  processing  of  the  instructional  materials 
and  sequences.  Although  a  good  deal  of  laboratory  research  has  been  directed  at  this 
area,  these  findings  are  not  directly  translatable  to  applied  instruction.  Research  in  such 
areas  as  mental  imagery,  human  information  processing  attention,  and  organization  in 
memory  can  be  useful  to  CAI  design,  if  suitable  transformations  to  applied  problems  can 
be  specified.  According  to  the  author,  the  chief  means  for  currently  influencing  the 
students,  learning  is  through  the  sequencing  of  instruction.  Use  of  the  findings,  models, 
and  theories  from  the  laboratory  can  potentially  expand  the  control  of  learning. 

Rigney,  J.  W.,  6c  Towne,  D.  M.  Computer-aided  performance  training  for  diagnostic  and 
procedural  tasks.  The  Second  Symposium  on  Cost  Effective  Learning  Through  the 
Application  of  Computer  Technology,  Washington,  DC,  February  1973,  179-207. 

The  authors  have  given  an  example  of  the  use  of  CAI  in  diagnostic  and  procedural 
tasks  through  their  training  computer  program  TASKTEACH.  The  specific  examples  of 
software  and  hardware  used  in  conjunction  with  TASKTEACH  are  explicated  along  with  a 
lengthy  list  of  computer  subroutines. 


(equipment)  are  higher.  Operating  at  full  capacity,  the  PLATO  system  is  cost  effective. 
Potential  applications  need  to  be  explored. 

Lahey,  G.  F.,  Hurlock,  R.  E.,  Sc  McCann,  P.  H.  Post  lesson  remediation  and  student 
control  of  branch  in  computer  based  training  (Tech.  Rep.  SRR  73-19).  San  Diego: 
Naval  Personnel  and  Training  Research  Laboratory,  April  1973. 

Training  time  and  final  examination  performance  are  as  good  when  students  select 
their  own  training  material  as  when  selection  of  materials  is  controlled  by  the  program. 
Also,  students  in  a  no-remediation  control  group  do  as  well  as  students  receiving 
remediation  after  failing  to  pass  lesson  tests.  Students  prefer  student  controlled  training. 

Landa,  S.  CATTS:  Computer-aided  training  in  troubleshooting  (Tech.  Rep.  R-518-PR). 
Santa  Monica:  Rand  Corporation,  May  1972. 

This  report  examines  computer-aided  instruction  (CAI)  as  an  efficient  and  effective 
teaching  medium.  CAI  offers  the  potential  to  decrease  number  of  instructors,  provide 
individualized  instruction,  and  reduce  long  term  equipment  costs.  Yet,  most  CAI  systems 
have  some  limitations  such  as  difficulty  in  using  the  graphical  capabilities  of  CRT  through 
course  author  languages.  The  author  of  this  report  describes  on-going  programs  at  Rand 
that  permit  direct  and  easy  creation  of  graphic  displays  on  a  CRT.  The  Programmer- 
Oriented  Graphics  Operation  (POGO)  was  used  to  train  airmen  in  malfunction  diagnoses. 
The  initial  tryout  of  the  system  proved  quite  successful.  The  problem  structure  of  the 
prototype  course  provided  a  convenient  framework  for  classroom  presentation.  The  high 
cost  of  hardware  necessary  to  support  a  graphics  system  is  largely  offset  by  resulting 
economies  in  instruction  and  teaching  efficiency. 

Mathis,  A.,  Smith,  T.,  Sc  Hansen,  D.  College  students*  attitudes  toward  computer-assisted 
instruction  (Unpublished  manuscript^  Tallahassee:  Florida  State  University,  1969. 

This  study  attempted  to  determine  what  aspects  of  the  learning  experience  might 
influence  student  attitudes  toward  computer-assisted  instruction  (CAI).  Sixty-four 
randomly  selected  students  were  given  CAI  or  reading  materials  on  general  psychology. 
Half  saw  familiar  material,  half  saw  unfamiliar  material.  Half  were  pretested  for  prior 
attitudes  toward  CAI.  The  analysis  showed  that  pretesting  had  no  effect.  Student 
attitudes  toward  CAI  were  influenced  more  by  actually  experiencing  CAI  than  by 
readings.  A  significant  factor  in  determining  the  positiveness  of  a  student's  reaction  to 
CAI  was  the  course  taken.  The  magnitude  of  the  change  in  attitude  was  dependent  upon 
the  kind  of  experience  the  student  had.  Those  students  who  were  given  familiar, 
immediately  relevant  material  showed  most  increase  in  positiveness.  Those  students  who 
made  more  errors  were  less  favorable  to  CAI.  These  results  indicate  that  it  may  be 
possible  to  convert  students  to  CAI  as  a  method  of  instruction  if  they  are  given  relevant 
programs  that  do  not  allow  them  to  make  too  many  errors. 

Mayo,  G.  D.  Computer  managed  instruction.  The  Second  Symposium  on  Cost  Effective 
Learning  Through  the  Application  of  Computer  Technology,  Washington,  DC,  February 
1973,  134-140. 

Computer-managed  instruction  (CMI)  is  defined  and  a  few  examples  where  training 
efficiency  was  dramatically  increased  throught  the  use  of  CMI  are  given.  A  scientific 
evaluation  of  three  instructional  modes  is  proposed:  (1)  a  group  instructed  by  CMI;  (2)  a 
group  instructed  by  individually-paced  instruction  with  minimal  computer  support;  and  (3) 
a  group  instructed  by  conventional  classroom  procedures.  This  study  should  delineate 

more  clearly  the  relative  advantages  of  these  modes  of  instruction  and  the  parameters  of 
training  efficiency  and  cost. 
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techniques.  The  authors  of  that  project  were  Ford,  Slough,  and  Hurlock  (1972)  under  the 
auspices  of  the  naval  Basic  Electricity/Electronics  School  (BE/E).  Five  CAI  modules  were 
developed  and  implemented  at  the  (BE/E)  school  during  this  project.  Comparison  of  the 
comparative  results  between  the  CAI  modules  and  with  the  traditional  instruction  yielded 
these  favorable  conclusions:  (1)  CAI  provided  very  effective  training.  (2)  CAI  students 
scored  consistently  higher  than  students  given  conventional  classroom  training.  (3)  CAI 
training  was  very  efficient  since  45-percent  less  training  time  was  required.  (4)  Student 
reactions  to  CAI  programs  were  predominantly  favorable  with  a  few  exceptions.  (5) 
Students  preferred  to  take  the  bulk  of  their  training  by  CAI. 

Hansen,  D.  N.,  Kribs,  D.,  Johnson,  B.,  <Jc  Dick,  W.  Office  of  Naval  Research  sponsored 
project  on  computers  and  instruction.  Tallahassee:  Computer-Assisted  Instruction 
Center,  September  1973. 

Florida  State  University  conducted  a  five-year  study  on  learning  strategies,  training 
strategies,  validation  strategies,  and  computer  system  strategies.  Learning  strategy 
research  focused  on  the  interaction  of  adaptive  instructional  design  with  task  and  learner 
characteristics.  Computer-managed  instruction  was  the  central  area  in  training  strategy. 
Sequential  and  tailored  testing  research  was  also  covered.  The  results  were  analyzed  in 
relation  to  ongoing  Navy  training  methods,  existing  constraints,  and  future  requirements. 
Computer  system  strategies  included  interactive  systems,  data  analysis  development,  and 
management  strategies. 

Hurlock,  R.  E.  Applications  of  pretest  branching  designs  to  CAI  basic  electronics  training 
(Tech.  Rep.  SRR  73-8).  San  Diego:  Naval  Personnel  and  Training  Research  Laboratory, 
September  1972. 

Pretest  branching  strategies  intended  to  adapt  instruction  to  individual  training  needs 
were  incorporated  into  a  CAI  system.  Compared  to  fixed  sequence  lessons,  pretest 
lessons  allowed  a  much  greater  range  in  training  time. 

Hurlock,  J.  A.,  <5c  Hurlock,  R.  E.  Comparision  of  paired  students  and  individual  students 
trained  by  CAI  (Tech.  Rep.  SRR  73-21).  San  Diego:  Naval  Personnel  and  Training 
Research  Laboratory,  June  1973. 

An  experiment  was  done  to  determine  if  existing  CAI  programs  could  train  paired 
students,  as  well  as  individuals.  The  results  showed  that  paired  student  training  was  as 
effective  as  individual  training  and  reduced  the  cost  per  terminal  hour.  CAI  course 
materials  was  basically  linear  and  students  were  paired  on  the  basis  of  learning  rate  and 
aptitude. 

Hurlock,  R.  E.,  &  Slough,  D.  A.  Experimental  evaluation  of  PLATO  IV  technology;  Final 
report  (NPRDC  Tech.  Rep.  76TQ-44).  San  Diego:  Navy  Personnel  Research  and 
Development  Center,  August  1976. 

The  results  of  eight  studies  designed  to  evaluate  the  PLATO  IV  system  of  computer- 
based  instruction  for  various  content  areas  are  summarized.  Techniques  investigated 
included:  use  of  still,  animated,  and  interactive  graphics;  simulation;  drill  and  practice; 
feedback  in  the  context  of  games;  guidance  of  offline  instruction;  learner  control  of 
instruction;  and  use  of  preprogrammed  instructional  strategies.  Measures  of  cost  and 
training  effectiveness  were  taken  for  the  development  and  delivery  of  instruction.  The 
system  was  found  to  be  effective  in  training  a  variety  of  cognitive  and  procedural  skills. 
The  interactive  graphic  display  and  touch  panel  is  an  innovative  and  cost  effective 
approach  to  simulation.  Course  development  costs  are  reasonably  low,  but  delivery  costs 
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Key  features  of  the  CAI  course  were:  (1)  self  paced,  (2)  hands-on  practical  exercises 
conducted  in  the  student  carrels,  and  (3)  additional  hands-on  practical  exercises  using 
larger  equipment  conducted  away  from  the  carrels. 

Summative  evaluation  of  the  implication  of  the  CAI  course  yielded  the  following 
results:  (1)  higher  achievement  for  CAI  group;  (2)  retention  and  transfer  enhanced  by 
CAI;  (3)  failure  rate  reduced  by  21  percent  for  CAI  group;  (4)  training  time  reduced  by  35 
percent  for  CAI  group;  and  (5)  highly  favorable  student  attitude  toward  CAI. 

The  interest  in  CAI  engendered  by  these  results  prompted  the  Continental  Army 
Command  (CONARC)  to  establish  a  task  study  group.  The  major  findings  of  the  study 
group  were:  (1)  Most  technical  courses— especially  those  with  one-to-one  student- to- 
equipment  ratios- -are  suitable  for  presentation  by  CAI.  (2)  Nontechnical  courses  are 
equally— if  not  more— suitable  for  presentation  by  CAI.  (3)  CAI  is  cost  effective  in 
comparison  to  conventional  instruction  when  the  student  load  is  high  enough  to  offset  the 
added  costs  of  buying  or  leasing  computer  hardware  and  software.  (4)  The  most  plausible 
hardware  configuration  for  future  CAI  efforts  in  the  Army  is  one  that  makes  use  of  the 
multiprocessor,  minicomputer  concept.  (5)  The  intangible  benefits  from  the  application  of 
CAI  are  as  important  as  those  that  can  be  measured  empirically. 

Dick,  R.  A.,  Royal,  J.  W.,  Simpson,  H.  K.,  &  Mackie,  R.  R.  Personnel  technology;  Using 
an  inexpensive  computer-based  system  to  teach  performance-oriented  skill  (Tech.  Rep. 
1728-1).  Arlington:  Personnel  and  Training  Research  Programs,  June  1974. 

Computer-based  and  programmed  textbook-based  curricula  for  training  students  were 
compared.  The  curricula  were  identical  except  that  students  in  the  computer-based 
course  received  simulation  training  and  students  in  the  textbook-based  course  used 
diagrams  and  written  explanations.  Although  the  computer-based  curriculum  seemed  to 
be  more  stimulating,  no  advantage  in  actual  learning  was  observed  for  the  computer 
trained  group. 

Fletcher,  3.  D.  Computer  application  in  education  and  training  status  and  trends  (NPRDC 
Tech.  Rep.  75-32).  San  Diego:  Navy  Personnel  Research  and  Development  Center, 
April  1975. 

Major  developments  in  the  military  and  civilian  sectors  are  described  in  computer- 
based  training,  with  emphasis  on  computers  for  teaching  and  instruction  and  on  new 
developments  with  implication  for  future  Navy  training.  The  operational  implementation 
of  computer-based  training  is  proceeding  more  rapidly  than  the  assessments  of  their 
economic,  technological,  and  pedagogical  impact.  Much  attention  is  presently  directed  at 
computer-managed  instruction,  use  of  models,  student  control,  authoring  and  author 
languages,  and  cost  of  material  preparation  costs.  Simultaneously,  the  Navy  has  defined 
its  training  system  characteristics.  Systems  must  lead  to  decreased  costs,  be  portable,  be 
relatively  instructor  free,  and  be  distributed  across  Navy  locations.  Therefore,  instruc¬ 
tion  must  be  flexible  and  modular.  Advances  in  computer  technology  are  making  this 
possible.  If  technology  advances  are  coupled  with  systematic  long  term  planning, 
computer  based  training  offers  major  economies  and  efficiencies  in  the  future. 

Ford,  J.  D.,  3r.  CAI  for  Navy  technical  training.  The  Second  Symposium  on  Cost 
Effective  Learning  Through  the  Application  of  Computer  Technology,  Washington,  DC: 
February  1973,  230-242. 

Much  of  the  information  presented  in  this  paper  came  from  a  project  to  develop  CAI 
techniques  and  methods  and  to  test  the  CAI  training  modules  derived  from  those 
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performance.  The  study  focused  on  the  selection  of  individual  difference  measures, 
selection  of  training  topics,  development  of  experimental  courses,  and  development  of 
criterion  measures.  The  most  compelling  effect  was  the  interactive  relationship  between 
learner  anxiety  and  the  method  of  instruction.  However,  most  relationships  among  major 
variables  did  not  show  strong  effects.  The  authors  believe  that  noncognitive  learner 
characteristics  have  not  received  adequate  evaluation. 

Taylor,  J.  E.,  Montague,  E.  K.,  <5c  Hanke,  R.  The  interrelationships  of  ability  level, 
instruction  system,  and  skill  acquisition  (Tech.  Rep.  29-70).  Alexandria:  Human 
Resources  Research  Organization,  September  1970. 

The  Army's  training  population  is  large  and  heterogeneous.  This  project  assessed  the 
impact  of  these  extreme  aptitude  differences  upon  learning  performance  in  Army 
noncombat  and  combat  skills,  and  undertook  the  development  of  instructional  strategies 
to  optimize  training  efficiency  for  men  of  differing  aptitude  levels.  High  aptitude 
learners  seem  to  learn  most  efficiently  when  left  alone  with  the  instructional  objective 
and  the  necessary  materials.  The  middle  aptitude  trainees  appear  to  learn  most 
efficiently  if  they  have  a  person  of  authority  of  whom  they  can  ask  questions.  Low 
aptitude  trainees  require  complete  structuring  of  course  material  and  an  instructor  to 
give  prompting  and  feedback.  The  high  variability  in  performance  among  low  aptitude 
trainees  was  examined. 


Valverde,  H.  H.  Effect  of  confirmation  peeking  and  response  mode  on  programmed 
instruction  (Tech.  Rep.  67-225).  W right-Pa tterson  Air  Force  Base,  OH:  Aerospace 
Medical  Research  Laboratory,  April  1969. 

An  experiment  was  done  to  test  the  effect  on  learning  efficiency  of  different 
response  modes  in  programmed  texts.  Two  response  modes  were  compared,  a  covert  mode 
which  encouraged  peeking,  and  an  overt  mode  which  did  not  permit  peeking.  Programmed 
texts  on  radar  orientation  were  administered  to  78  Air  Force  pilot  trainees.  The  rate  at 
which  the  subjects  completed  the  program  was  the  measure  of  efficiency.  No  statistical 
difference  in  efficiency  was  found  between  the  overt  and  the  covert  group. 

3.  Computer  Assisted/Managed  Instruction. 

Charp,  S.  Use  of  the  computer  for  individualization  of  instruction.  The  Second 
Symposium  on  Cost  Effective  Learning  Through  the  Application  of  Computer  Technol¬ 
ogy,  Washington,  DC:  February  1973,  307-322. 

The  results  of  introducing  computer  assisted  and  managed  instruction  (CAI  and  CMI) 
into  the  Philadelphia  School  System  as  instructional  alternatives  in  general  math,  algebra, 
reading,  biology,  and  electronics  are  described.  The  increased  effectiveness  and  time 
savings  of  CAI  and  CMI  seem  to  point  toward  their  expanded  role  in  the  educational 
system  at  large.  However,  high  courseware  development  costs  may  markedly  constrain 
that  growth. 

Cieri,  V.  P.  Development  of  a  computerized  training  system  for  future  army  training. 
The  Second  Symposium  on  Cost  Effective  Learning  Through  the  Applicaton  of  Computer 
Technology,  Washington,  DC:  February  1973,  212-229. 

The  author  summarizes  the  development  and  evaluation  of  a  tutorial  mode  computer- 
assisted  instruction  (CAI)  project  developed  for  use  at  the  U.S.  Army  Signal  Center  and 
School. 


instructional  strategies.  Recent  observers  have  suggested  that  a  more  eclectic  approach 
to  instructional  design  be  followed.  Such  an  approach  requires  that  several  small-scale 
theories  or  models  be  developed  to  specify  the  students'  internal  processing  modes  and, 
thus,  serve  as  a  basis  for  instruction.  After  this  brief  sketch  of  the  role  of  theory,  Powers 
examines  the  interaction  of  learner  traits  with  instructional  strategies.  Research  on 
adaptive  training  suggests  two  important  conclusions:  (1)  Learner  traits  may  interact 
with  may  variables  in  addition  to  instructional  technique.  (2)  Many  of  the  traits  upon 
which  instruction  is  selected  (e.g.,  general  intelligence)  may  in  fact  be  insensitive 
indicators  of  adaptive  instruction.  More  pragmatic  approaches  to  adaptive  instruction  are 
required  to  take  into  account  the  several  important  variables  that  might  impact  upon  type 
of  instruction.  Early  application  of  adaptive  training  techniques  did  little  more  than  pace 
the  rate  of  instruction  for  the  individual.  Such  factors  as  content  and  sequence  of 
instruction  may  provide  better  modulation  of  learning.  Furthermore,  adaptation  of 
instruction  based  on  within- task  performance  offers  improved  impact  of  the  adapted 
instruction. 


Rankin  W.  C.,  &  Hall,  E.  R.  Demonstration  of  a  methodology  for  classifying  naval 
training  courses  (TAEG  Rep.  No.  51).  Orlando:  Training  Analysis  and  Evaluation  Group, 
November  1977. 


This  report  documents  the  development  of  a  methodology  for  classifying  naval 
training  courses.  Data  were  gathered  from  over  400  Navy  enlisted  technical  training 
courses  and  analyzed  by  a  computer-based  clustering  algorithm  into  homogeneous 
subgroupings  based  on  similiarities  among  courses.  The  ultimate  goal  was  to  provide 
training  management  with  an  information  structure  to  support  decisions  about:  (1) 
instructor  requirements,  (2)  training  media  and  equipment,  (3)  training  facilities,  (4)  kinds 
of  skills  trained,  and  (5)  strategies  for  training  and  evaluation.  This  approach  is  based  on 
a  numerical  taxonomy  that  permits  natural  categories  to  emerge,  rather  than  precon¬ 
ceived  notions  of  what  constitutes  the  categories.  Hypothetical  data  are  used  only  to 
illustrate  how  the  clustering  program  works. 


Tallmadge,  G.  K.,  Shearer,  3.  W.,  &  Greenberg,  A.  M.  Study  of  training  equipment  and 
individual  differences:  The  effect  of  subject  matter  variables  (Tech.  Rep. 
NAVTRADEVCEN  67-C-01 14-1).  Orlando:  Naval  Training  Device  Center,  May  1968. 


This  report  was  aimed  at  defining  instructional  methods  that  differ  in  design  and  use 
as  functions  of  learner  characteristics.  Two  subject  matter  areas  were  selected:  logico- 
mathematical  procedures  and  visual- form  discrimination.  Two  separate  courses  were 
developed  for  each  subject  matter  are?..  One  approach  was  inductive  and  the  other  was 
deductive.  Furthermore,  each  student  was  tested  on  28  measures  of  aptitude,  interest, 
and  personality  variables.  The  results  of  the  study  support  the  existence  of  individualized 
learning  styles  and  the  use  of  multitrack  instruction  as  a  cost  effective  means  of 
improved  learning.  The  report  also  includes  a  summary  of  style  literature.  The  survey 
considered  both  cognitive  and  noncognitive  characteristics  of  learners.  The  important 
role  of  subject  matter  content  is  often  overlooked  in  analyses  of  interactions  between 
learning  styles  and  instructional  methods. 


Tallmadge,  G.  K.,  ic  Shearer,  3.  W.  Study  of  training  equipment  and  individual 
differences:  Research  on  interactive  relationships  among  learner  characteristics,  types 
of  learning,  instructional  methods,  and  subject  matter  variables.  Orlando:  Naval 
Training  Device  Center,  3 une  1970. 


A  model  was  developed  for  simultaneously  examining  the  effects  of  learner  charac¬ 
teristics,  types  of  learning,  instructional  methods,  and  subject  matter  variables  on  student 


forms  of  adaptive  systems;  one  in  which  the  performance  measurement  and  the  adaptive 
variable  change  simultaneously,  and  another  in  which  the  rate  of  change  of  the  adapative 
variable  changes  without  a  progressive  change  in  the  performance  measurement  during 
adaptation.  In  implementing  any  adaptive  training  system,  the  three  essential  elements  in 
the  adaptive  trainer  must  be  considered,  and  a  series  of  decisions  must  be  made  with 
respect  to  each.  When  these  choices  are  made  skillfully  and  tested  in  the  process,  the 
training  system  can  be  much  more  effective  as  a  result  of  having  been  made  adaptive. 

Koran,  M.  L.  Individual  differences  in  information  processing:  Comments  on  the  paper  by 
E.  Hunt.  In  R.  C.  Anderson,  R.  3.  Spiro,  &  W.  E.  Montague  (Eds.),  Schooling  and  the 
Acquisition  of  Knowledge:  Conference  proceedings  (NPRDC  Tech.  Rep.  78-6).  San 
Diego:  Navy  Personnel  Research  and  Development  Center,  December  1977. 

Research  findings  in  the  basic  psychology  of  individual  differences  and  in  the  area  of 
learning  have  never  been  systematically  combined  although  the  results  of  such  a 
combination  might  be  used  to  adapt  instruction  to  individual  students.  Attempts  to 
individualize  instruction  have  been  based  on  hypothesized  interactions  between  aptitude 
and  treatment  variables.  Perhaps,  initial  research  has  been  inconclusive,  because  the 
cognitive  processes  of  aptitude  and  kinds  of  treatment  were  not  considered.  Current 
aptitude  test  items  are  chosen  for  their  power  to  predict  performance  in  our  standardized 
educational  programs  and  not  for  their  relationships  to  potential  intellectual  processes. 
Measures  need  to  be  designed  to  tap  different  ways  of  learning.  A  closer  analysis  of 
cognitive  processes  may  produce  more  effective  mechanisms  for  instructional  research 
into  aptitude- treatment  interactions. 

Landa,  L.  N.  Algorithmization  in  learning  and  instruction.  Englewood  Cliffs,  N3: 
Educational  Technology  Publications,  1974. 

An  algorithm  is  a  method  for  carrying  out  the  elementary  operations  necessary  to 
solve  a  problem.  The  basic  subject  matter  of  this  book  is  the  application  of  algorithmic 
methods  to  education  and  psychology.  The  characteristics  of  algorithmic  methods  are 
contrasted  with  nonalgorithmic  methods  of  problem  solving.  Theoretical  problems  in 
teaching  algorithms  are  discussed.  The  logical  and  psychological  problems  in  constructing 
algorithms  of  identification  are  considered,  and  mathematical  methods  in  the  analysis  and 
evaluation  of  algorithms  are  presented.  Research  findings  are  discussed. 

Modrick,  3.  A.,  Levit,  R.  A.,  Alden,  D.  G.,  ic  Henke,  A.  3.  Review  of  approaches  to 
cognitive  style  and  implications  for  human  information  processing  in  command  and 
decision  situations.  Proceedings  of  the  Human  Factors  Society  19th  Annual  Meeting, 
Dallas,  October  1975,  333-335. 

This  paper  defines  cognitive  style  as  an  individual's  preferred  and  consistent  way  of 
organizing  informal.. on  and  translating  the  processed  information  into  overt  action.  There 
are  a  limited  number  of  different  cognitive  styles,  each  with  an  associated  strategy  for 
problem  solving.  Representative  concepts  are  summarized.  The  usefulness  of  the  concept 
of  cognitive  style  is  discussed. 

Powers,  T.  E.  Instructional  strategies  and  individual  differences:  A  selective  review  and 
summary  of  literature.  Princeton,  N3:  Educational  Testing  Service,  September  1976. 
(AD-A040  961) 


The  author  begins  with  a  review  of  the  major  theories  that  impact  upon  instructional 
design.  Recent  theoretical  review  has  suggested  that  traditional  theories  of  learning 
(e.g.,  Thorndike  it  Skinner)  do  not  provide  accurate  guidance  for  development  of 


Fox,  W.  L.,  it  Taylor,  3.  E.  Adaptation  of  training  to  individual  differences.  Alexandria: 

Human  Resources  Research  Organization,  September  1967. 

The  U.S.  Army  has  to  train  men  of  different  aptitude  levels  in  a  wide  variety  of  jobs. 
The  objective  of  the  research  is  to  develop  procedures  for  achieving  effective  training  at 
all  aptitude  levels  for  men  in  high-density  military  jobs.  The  15  tasks  chosen  for 
investigation  had  (1)  elements  in  common  with  the  skills  and  knowledge  required  in  the 
majority  of  military  jobs  and  (2)  required  skill  levels  that  could  be  classified  from  simple 
to  complex.  Subjects  were  tested  from  a  high  aptitude  group,  a  middle  aptitude  group, 
and  a  low  aptitude  group,  as  determined  by  AFQT  scores.  Training  procedures  were 
tailored  to  give  the  lower  aptitude  trainee  every  advantage.  The  results  indicate  that 
learning  performance  is  directly,  and  highly,  related  to  aptitude  level  across  tasks  and 
complexity  levels.  Performance  was  found  to  be  related  to  training  methods  for  both  high 
and  low  level  group>s.  Performance  of  individuals  was  found  to  be  highly  consistent  across 
tasks.  The  low  aptitude  trainee  can  learn  to  perform  a  variety  of  tasks  if  the  training 
methods  have  been  carefully  selected  and  organized  to  ensure  his  assimilation  of  the 
instruction. 


Gopher,  D.  Williges,  B.  H.,  Williges,  R.  C.,  6c  Damos,  D.  L.  Manipulating  the  number  and 
type  of  adaptive  variables  in  training  (Tech.  Rep.  ARL-74- 1 2).  Urbana-Champaign,  IL: 
Avionics  Research  Laboratory,  October  1974. 

Interest  in  automatically  adaptive  techniques  is  increasing,  because  such  techniques 
might  provide  an  optimum  learning  model  to  teach  psychomotor  skills.  This  study  was 
designed  to  investigate  the  implications  of  adapting  stimulus  and  response  variables,  the 
effect  of  varying  the  number  of  variables,  and  the  usefulness  of  adaptive  techniques  in 
training  for  transfer  to  changing  task  conditions.  Data  were  collected  and  analyzed  for 
each  adaptive  variable.  Results  are  discussed  in  terms  of  stimulus  and  response 
similarity,  the  optimum  number  of  adaptive  variables,  and  the  appropriateness  of  a 
changing  task  to  evaluate  adaptive  training. 

Hogan,  3.  C.  Trainability  of  abilities:  A  review  of  nonspecific  transfer  issues  relevant  to 
ability  training  (Tech.  Rep.  ARRO  3010-TRl).  Washington,  DC:  Advanced  Research 
Resources  Organization,  3anuary  1978. 

A  literature  review  was  conducted  to  investigate  the  possibility  of  improving  general 
abilities  by  training  related,  but  not  identical  tasks.  Areas  reviewed  include  early 
memory  training  research,  verbal  and  motor  studies  of  warmup  and  learning  to  learn, 
effects  of  practice  variability  and  learning  without  a  prototype,  and  applied  and 
educational  training.  Some  indirect  support  for  the  feasibility  of  nonspecific  transfer  can 
be  drawn  from  several  areas  of  research,  including  simulator  training  and  educational 
training.  Further  empirical  testing  is  necessary.  The  implications  of  nonspecific  transfer 
and  ability  training  are  discussed.  These  implications  include  the  possibility  of  raising  the 
ability  levels  of  individuals  whose  work  requires  specific  abilities  and  increasing  the 
flexibility  of  personnel  assignment  in  jobs  through  training. 

Kelley,  C.  R.  What  is  adaptive  training?  Paper  presented  at  the  meeting  of  the 
New  England  Psychological  Association,  November  1968. 

Adaptive  training  is  training  in  which  the  problem- -the  stimulus  or  the  task — is 
varied  as  a  function  of  how  well  the  trainee  performs.  Performance  must  be  continuously 
or  repetitively  measured  in  some  way  and,  then,  used  to  make  appropriate  changes  in  the 
stimulus,  problem,  or  task.  An  adaptive  trainer  has  three  essential  elements:  the 
adaptive  variables,  the  performance  measurement,  and  the  adaptive  logic.  There  are  two 


Hammell,  T.,  <Sc  Mara,  T.  D.  Application  of  decision  making  and  team  training  research  to 
operational  training:  A  translative  technique  (Tech.  Rep.  NAVTRADEVCEN  68-C- 
0242).  Groton,  CT:  Naval  Training  Device  Center,  April  1970. 

The  present  study  developed  and  applied  a  technique  for  translating  laboratory 
research  findings  in  decision  making  into  forms  of  use  in  training  systems.  The  technique 
involved  a  decision  skills  taxonomy  and  a  behavioral  deficiency  taxonomy.  Research 
findings  which  were  categorized  using  the  two  taxonomies,  were  useful  in  translated  form 
for  predicting  the  behavioral  deficiencies  that  might  be  associated  with  operational 
decision  tasks.  The  applicability  of  the  translated  research  findings  to  training  and  real 
world  systems  was  evaluated  by  analyzing  submarine  attack  simulator  (21A37/4)  and 
actual  at-sea  performance  data.  A  method  was  devised  for  applying  for  the  translated 
research  findings  to  decision  making  training. 

Kribs,  H.  D.,  Thurmond,  P.,  &  Mark,  L.  Computerized  collective  training  for  teams  (Tech. 
Rep.  TR-77-A4).  Alexandria:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  February  1977. 

This  report  provides  an  initial  framework  for  the  development  of  instructional 
strategies  for  computer-assisted  team  training.  This  framework  is  based  on  a  state-of- 
the-art  assessment  of  instructional  strategies  and  theories.  Two  conclusions  are  drawn: 
(1)  At  present  there  is  no  framework  for  a  set  of  general  purpose  instructional  strategies 
and  (2)  an  instructional  system  development  approach  must  be  developed  for  team 
training.  The  initial  framework  developed,  which  requires  much  refinement  and  verifica¬ 
tion,  is  based  on  discussion  and  analysis  of  three  factors  underlying  an  instructional 
system:  the  characteristics  of  a  team  and  of  team  tasks,  the  characteristics  of  the 
learner  and  of  learner  strategies,  and  the  capabilities  of  computer-assisted  instruction. 

2.  Adaptive/Individualized  Training. 

Charles,  J.  P.,  &  Johnson,  R.  M.  Adaptive  training  demonstrations- -lessons  learned. 
Proceedings  of  the  Sixth  Naval  Training  Equipment  Center  and  Industry  Conference. 
Orlando,  November  1973,  449-459. 

The  technical  feasibility  of  using  automated  adaptive  training  techniques  in  conjunc¬ 
tion  with  instrument  flight  maneuver  training  is  examined.  Eight  conclusions  were 
derived  from  the  results  of  the  study.  (1)  Student  performance  must  be  quantified  and 
summarized  for  automated  adaptive  instruction  to  be  applied.  (2)  Although  ultimate 
performance  measures  may  not  be  available,  existing  measures  are  sufficient  for 
successful  application  of  adaptive  principles.  (3)  Normalizing  and  averaging  performance 
data  provide  adequate  resolution  for  training  control.  (4)  Introduction  of  artificial 
adaptive  variables  is  probably  not  necessary,  as  sufficient  analysis  seems  to  reveal 
appropriate  variables  in  all  flight  missions.  (5)  Adaptive  training  allows  students  to 
progress  at  their  own  speeds.  (6)  Adaptive  training  can  enhance  student  motivation.  (7) 
computer-controlled  speech  can  successfully  provide  feedback  to  students.  (8)  Simple, 
adaptive  algorithms  can  be  applied  successfully  to  modulate  trial-by-trial  performance. 
The  benefits  realized  by  the  use  of  adaptive  training  techniques  are:  realizing  training 
economies,  increasing  student  motivation,  expanding  the  training  scope,  obtaining  objec¬ 
tive  performance  data,  and  relieving  instructors  of  some  burdens  thereby  allowing  more 
meaningful  interactions  with  students. 


team,  a  ship  control  team,  and  a  flight  deck  team.  From  this  descriptive  work  certain 
unexpected  problems  were  founds  (1)  The  boundaries  of  team  membership  were  never  as 
clearly  defined  as  anticipated.  (2)  Teams  undergo  marked  changes  in  composition  and 
activity  when  the  problems  they  face  change.  Various  measures  and  different  strategies 
were  used  to  summarize  the  network  of  interactions  between  team  members.  The  finding 
that  team  performance  is  almost  entirely  a  chain  of  individual  performance  has 
implications  for  training. 

This  paper  also  examines  the  specific  difficulties  of  team  training,  including  the 
relative  absence  of  satisfactory  proficiency  measures  and  the  need  for  a  team  to  be  able 
to  correct  its  own  errors. 

Finally,  a  literature  review  was  conducted  to  find  relevant  information  on  group 
behavior  and  training  problems.  The  areas  covered  include  the  dimensions  of  groups;  the 
relation  of  attitudes,  perception,  and  motivation  to  group  performance;  the  use  of  special 
techniques  for  the  improvement  of  group  performance,  communication  networks;  and  the 
relation  of  individual  proficiency  to  group  performance. 

Hall,  E.  R.,  &  Rizzo,  W.  A.  An  assessment  of  U.S.  Navy  tactical  team  training  (TAEG 
Rep.  18).  Orlando:  Training  Analysis  and  Evaluation  Group,  March  1975.  (AD-AOH 
H52) 

The  report  was  prepared  to  provide  information  on  current  team  training  practices, 
problems,  and  research  to  be  used  for  planning  improved  team  training  practices.  Several 
team  training  sites  including  surface,  subsurface,  and  air  operations  were  visited  and  the 
relevant  technical  literature  was  reviewed  to  provide  the  basis  for  recommendation  of 
solutions  to  current  problems.  The  first  problem  encountered  in  examining  team  training 
was  determination  of  a  definition  of  "team."  The  following  characteristics  of  a  team 
were  synthesized  from  the  literature  review:  goal-or  mission-oriented,  formal  structure, 
assigned  roles,  and  interaction  among  members.  Another  term  relevant  to  team  training 
and  requiring  definition  is  tactical  decision  making.  Again,  the  authors  synthesized  the 
following  characteristics:  situation  diagnosis,  hostile  environment,  selection  of  optimum 
alternatives,  and  some  degree  of  uncertainty. 

An  important  consideration  in  team  training  is  the  extent  to  which  team  performance 
is  more  than  just  a  sum  of  individual  performances.  It  is  a  widespread  belief  that  this  is 
so,  but  the  relative  contribution  of  individual  performances  is  not  well  defined.  Although 
many  feel  that  team  coordination  should  be  trained,  the  meaning  of  the  term  is  unclear. 
Thus,  meaningful  training  objectives  and  methods  for  this  aspect  of  team  performance  are 
difficult  to  achieve. 


The  Navy  has  two  types  of  team  training  programs:  training  individuals  to  function 
as  team  members  and  training  the  intact  team.  The  relative  benefits  or  disadvantages  of 
these  two  approaches  have  not  been  fully  delineated.  Examination  of  current  naval  team 
training  practices  revealed  the  current  instructional  technology  is  not  applied.  Spec;fical- 
ly,  clear  objective  training  objectives  are  not  specified  in  most  cases,  and  teams  are  not 
provided  with  sufficient  performance  feedback.  The  authors  recommend  that  in  the 
future  more  emphasis  should  be  placed  on  individual  training  at  both  the  initial  and 
refresher  levels. 


Semple,  C.  A.  Guidelines  for  implementing  training  effectiveness  evaluations  (NTEC 
Tech.  Rep.  NAVTRAEQUIPCEN  72-C-0209-3).  Orlando:  Naval  Training  Equipment 
Center,  April  1974. 

This  report  describes  problems  encountered  in  field  studies  of  training  effectiveness. 
Procedures  are  presented  for  planning,  implementing,  and  documenting  training  effective¬ 
ness  evaluations.  Such  evaluations  should  provide  quantitative  and  qualitative  analyses  of 
the  effects  of  training  methods  upon  student  performance,  cost  of  achieving  benefits  in 
student  performance,  and  impacts  upon  other  training  resources.  Basic  assumptions 
include:  existing  training  objectives  are  complete  and  objective  changes  to  the  training 
situation  are  relatively  permanent,  and  baseline  data  exist  on  student  performance  levels. 
Training  effectiveness  evaluations  can  range  from  qualitative  to  carefully  designed 
transfer  of  training  studies. 

Sheldon,  M.  S.  Evaluation  of  training  in  a  simulated  environment.  In  3.  E.  Uhlaner  (Ed.), 
Psychological  Research  in  National  Defense  Today  (Tech.  Rep.  S-l).  Washington,  DC: 
U.S.  Army  Behavioral  Science  Research  Laboratory,  June  1967. 

This  paper  discusses  the  development  of  measurable  criterion  variables  to  assess 
empirical  crew  performance  and  considers  some  of  the  situational  variables  that  must  be 
part  of  the  simulated  exercise  to  extract  meaningful  measures. 

Instructional  Delivery  Systems 

1.  Team  Training. 

Biel,  W.  C.  Planning  for  team  training  in  the  system.  In  3.  E.  Uhlaner  (Ed.),  Psychological 
Research  in  National  Defense  Today  (Tech.  Rep.  S-l).  Washington,  DC:  U.S.  Army 
Behavioral  Science  Research  Laboratory,  3une  1976. 

This  paper  describes  the  various  general  phases  in  the  development  process  of  an 
operational  system  and  emphasizes  the  best  time  for  considering  team  training  require¬ 
ments.  The  phases  include  concept  phase,  project  definition  phase,  acquisition  phase,  and 
operational  phase.  Training  requirements  must  be  specified  in  the  early  concept  phase. 

Dieterly,  D.  L.  Team  performance:  A  model  for  research.  Proceedings  of  the  Human 
Factors  Society  22nd  Annual  Meeting,  Detroit,  October  1978,  486-492. 

The  author  briefly  characterizes  results  of  a  literature  review  on  team  performance 
as  emphasizing  interpersonal  interaction  variables.  Although  these  variables  are  seen  as 
important,  the  author  suggests  that  analysis  of  task  interactions  is  a  more  critical 
research  approach.  To  stimulate  this  research,  a  team  performance  model  is  proposed. 
This  model  contains  five  variables:  individual  capabilities,  task  function  design,  communi¬ 
cation,  control,  and  decision.  The  variable  of  task  function  design  is  emphasized  to 
indicate  one  of  several  bases  for  research  on  team  performance.  This  variable  refers  to 
the  tasks  and  task  interactions  that  may  be  defined  in  a  te*'  situation.  The  author 
maintains  that  this  is  a  relatively  neglected  approach  to  the  analysis  of  team  performance 
and  provides  a  good  starting  point  for  research  programs  on  team  performance. 

Glanzer,  M.  Experimental  study  of  team  training  and  team  functioning.  In  R.  Glaser 
(Ed.),  Training  Research  and  Education.  New  York:  Wiley,  1965. 

Data  were  gathered  and  analyzed  to  experimentally  describe  a  team.  Four  types  of 
Navy  teams  were  extensively  observed:  a  gunnery  team,  a  combat  information  center 
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Montague,  W.  E.  Quality  control  of  instruction:  A  logical  analysis  procedure.  Proceed¬ 
ings  of  the  Human  Factors  Society  22nd  Annual  Meeting.  Detroit,  October  1978,  91-97. 

The  author  begins  with  a  general  discussion  of  the  instructional  systems  development 
(ISD)  process  and  indicates  the  important  role  of  evaluation  of  this  process.  Although 
internal  evaluation  of  whether  training  objectives  are  met  and  external  evaluation  of  job 
performance  are  considered  important,  other  types  of  evaluation  are  suggested  to 
evaluate  the  early  phases  of  instructional  development.  The  instructional  quality 
inventory  (IQI)  was  developed  to  meet  these  needs.  This  inventory  provides  for  a  rational 
diagnosis  of  defects  in  instructional  materials.  The  IQI  is  intended  to  evaluate  materials 
along  two  dimensions:  consistency  among  all  materials  and  adequacy  with  respect  to 
instructional  technique.  The  primary  aspect  of  the  consistency  dimension  is  to  ensure 
scope,  level,  and  content  of  test  items  are  consistent  with  training  objectives.  Adequacy 
of  instructional  technique  is  determined  by  first  placing  instructional  presentation  into 
one  of  four  display  categories:  telling/presenting  a  definition,  telling/presenting  an 
example,  asking  the  definition  of  a  concept,  or  asking  about  an  example.  The  analyst 
assesses  the  adequacy  of  the  display  for  the  instruction  in  questions  along  four  dimensions: 
(1)  is  the  relevant  information  isolated  or  clearly  identified,  (2)  is  the  student  helped  or 
prompted,  (3)  are  examples  matched  to  new  situations  to  aid  learning  critical  features, 
and  (4)  is  there  sufficient  sampling  from  all  difficulty  levels.  Three  studies  were 
conducted  to  determine  the  validity  of  the  IQI  in  evaluating  instructional  materials.  In 
these  studies  instructional  materials  were  varied  to  affect  either  consistency  of  training 
objectives/test  items  or  adequacy  of  instructional  technique.  Results  confirmed  that 
prescriptions  from  the  IQI  are  actually  obtained  in  student  performance.  A  training 
manual  for  the  use  of  the  IQI  was  developed  and  evaluated  in  a  training  workshop  for 
analysts.  Results  indicated  that  users  could  use  the  IQI  reliably.  A  large-scale  validation 
study  is  in  progress. 

Rigney,  J.  W.,  Cremer,  R.  H.,  Towne,  D.  M.,  Bond,  N.  A.,  Jr.,  &  Mason,  A.  K. 
Measurement  and  predictions  of  cognitive  loadings  in  corrective  maintenance  tasks:  I, 
A  Bayesian  approach  (Tech.  Rep.  No.  46).  Los  Angeles:  University  of  Southern 
California,  Department  of  Psychology,  December  1965. 

This  report  describes  methods  for  analyzing  the  cognitive  loadings  involved  in 
electronic  troubleshooting  tasks.  A  symptom- malfunction  (S-M)  matrix  is  introduced 
showing  interrelationships  between  possible  malfunctions  and  sets  of  symptoms.  The 
Bayesian  Electronic  Trouble  Shooting  (BETS)  model  was  developed  as  a  criterion  measure 
of  troubleshooting  ability.  BETS  can  select  at  each  step  in  the  troubleshooting  sequence 
the  combination  of  test  and  test-point  that  has  the  greatest  potential  uncertainty 
reduction.  Data  were  collected  that  demonstrated  the  effectiveness  of  the  BETS  model  in 
troubleshooting.  The  BETS  model  has  potential  applications  in  teaching,  as  a  design  tool, 
and  as  a  criterion  measure  of  troubleshooting  ability. 

Salvendy,  G.,  &  Stewart,  G.  K.  The  prediction  of  operator  performance  on  the  basis  of 
performance  tests  and  biological  measures.  Proceedings  of  the  Human  Factors  Society 
19th  Annual  Meeting,  Dallas,  October  1975,  457-466. 

This  study  investigated  the  hypothesis  that  the  combination  of  biological  measures 
and  psychomotor  performance  test  scores  would  provide  a  more  valid  personnel  selection 
measure  than  just  a  psychomotor  performance  test  score.  Data  were  collected  using  the 
one-hole  test  elemental  performance  test  and  various  biological  measures.  It  was  found 
that  the  most  powerful  predictors  of  job  performance  were  the  absolute  values  of  the 
standard  duration  of  basal  interbeat  intervals.  The  hypothesis  that  the  combination  of 
measures  would  be  more  effective  than  a  single  measure  was  supported. 
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Program  Objective  Memorandum  (POM)  were  examined.  Weaknesses  in  these  procedures 
were  identified,  and  a  plan  of  action  for  their  improvement  was  proposed.  One  major 
weakness  identified  by  this  report  relates  to  the  generation  of  resource  requirement 
requests  (RRRs).  These  RRRs,  it  was  found,  are  based  largely  upon  speculative 
information  and  are,  at  best,  gross  approximations  of  the  requirements  which  will  actually 
be  faced  for  the  fiscal  year  being  planned  (a  2-  to  3-year  planning  horizon).  However, 
rather  than  treating  these  as  gross  approximations  and  allowing  flexibility  within  the 
system,  the  statement  of  RRRs  does  not  change  as  expectations  change.  Another  problem 
identified  is  that  the  Navy's  training  experts  do  not  participate  in  the  planning  process  at 
the  macro,  or  final,  decision  making  level  (e.g.,  determining  total  manpower  require¬ 
ments).  Similarly,  decision  makers  in  the  Washington  arena  cannot  view  the  training 
process  at  a  micro  level  (i.e.,  the  individual  course  level).  A  consequent  void  is  created  by 
this  lack  of  communication  betwen  micro-  and  macro-level  planners.  One  final  problem 
with  the  training  portion  of  the  POM  identified  in  this  report  is  the  absence  of  a  model  for 
use  during  final  planning  stages  to  examine  training  resource  impacts  upon  various 
decisions.  The  authors  propose  the  development  of  real-time  interactive  models  to  assess 
the  effects  of  alternatives  as  they  are  considered. 

In  the  appendices,  Lindahl  and  May  describe  and  identify  weaknesses,  and  make 
recommendations  pertaining  to:  (1)  the  generation  of  course  requirements,  (2)  the 
generation  of  student  input  requirements,  (3)  the  determination  of  instructor  require¬ 
ments,  (4)  the  determination  of  support  and  facility  requirements,  and  (5)  training 
execution.  Throughout  these  discussions,  the  need  for  increased  quantification  of  the 
design  process  and  commitment  to  systems  analytic  procedurs  for  instructional  design  and 
resource  requirement  planning  is  emphasized.  Also,  a  reexamination  of  many  assumptions 
which  are  used  in  planning  is  suggested. 

Meister,  D.  A  theoretical  structure  for  personnel  subsystem  measurement.  Proceedings 
of  the  22nd  Annual  Human  Factors  Society  Meeting,  Detroit,  October  1978,  474-478. 

This  paper  (1)  examines  ways  in  which  personnel  subsystem  measurement  (PSM) 
differs  from  the  controlled  environment  (CE)  measurement  characteristic  of  much  human 
factors  (HF)  research,  and  (2)  considers  ways  in  which  PSM,  which  is  a  measure  of 
personnel  performance  in  the  actual  work  environment,  is  more  useful  than  CE  in  HF 
research.  Various  types  of  PSM  are  differentiated.  The  author  contends  that  the 
normative  data  obtained  through  PSM  is  more  useful  at  this  stage  of  HF  development  than 
experimental  data,  since  what  is  needed  for  system  development  are  masses  of  descriptive 
data  that  will  enable  HF  psychologists  to  predict  how  systems  will  functions. 

Miller,  R.  B.,  6c  Duffy,  L.  R.  Design  of  training  systems:  The  development  of  scaling 
procedures  (Tech.  Rep.  T AEG- 3 2).  Orlando:  Training  Analysis  and  Evaluation  Group, 
December  1976.  (AD-A051  647) 

Miller  and  Duffy  explored  the  idea  of  applying  subjective  expected  utility  theory  to 
the  analysis  of  training  system  design  and  evaluation.  Their  basic  premise  was  that  the 
training  system  could  be  described  by  a  series  of  attributes  each  having  a  certain  utility. 
An  increase  in  the  value  of  some  attributes  would  improve  the  systems  utility,  while  an 
increase  in  other  attributes  would  decrease  the  systems  utility.  Considerable  develop¬ 
ment  and  justification  of  this  concept  was  described  including  a  statement  of  some 
potential  attributes  that  could  be  used.  These  concepts  could  have  significant  impact 
upon  assessment  of  a  training  program's  overall  desirability  (i.e.,  utility)  when  compared 
to  others. 


4.  Training  Devices. 

Brock,  J.  F.  Maintenance  training  and  simulation:  Design  processes  and  evaluation 
criteria.  Proceedings  of  the  Human  Factors  Society  22nd  Annual  Meeting,  Detroit, 
October  1978,  260-266. 

Four  aspects  of  training  unique  to  maintenance  task  training  are  identified  to  form 
the  basis  of  discussion:  (1)  content,  (2)  importance  of  decision  making,  (3)  importance  of 
understanding  system  relationships,  and  (4)  importance  of  knowing  a  wide  variety  of 
solution  sequences.  Three  kinds  of  instructional  requirements  for  maintenance  training 
are  identified:  (1)  procedural  (preventive  maintenance,  systems  checkout,  and  general 
inspection),  (2)  psychomotor  (calibration,  alignment,  and  repair),  (3)  schemata  (fault 
detection  and  troubleshooting).  Although  the  general  instructional  systems  development 
(ISD)  approach  to  training  is  the  same  for  maintenance  and  operation  jobs,  certain 
specifics  of  ISD  elements  must  be  tailored  for  the  job.  Of  these  three  kinds  of 
requirements,  the  training  of  schemata  is  the  most  complex,  least  understood,  and  least 
suited  to  current  ISD  approaches.  Recently,  the  armed  forces  have  begun  using  simulators 
for  maintenance  training.  Using  the  simulators  in  maintenance  training  has  proven  to  be 
both  effective  and  cost  effective.  Recent  advances  in  computer  technology  have 
enhanced  the  role  of  simulators  still  more.  Simulation  provides  the  following  advantages 
over  use  of  actual  equipment  in  maintenance  training:  cost,  reliability,  capability  of 
being  modified  for  hands-on  practice,  safety,  and  student  performance  as  good  or  better. 

The  author  lists  seven  simulation  options:  (1)  actual  equipment,  (2)  mockup,  (3)  3- 
D/2-D  mix  of  display,  (4)  3-D  schematic,  (5)  2-D/image  interface,  (6)  2-D/keyboard 
interface,  and  (7)  projected  displays.  The  problem  in  designing  maintenance  training 
programs  is  not  in  finding  techniques,  but  rather  in  determining  what  skills  to  be  taught. 
There  is  confusion  in  the  literature  concerning  the  distinction  between  test  reliability  and 
validity,  and  instructional  reliability  and  va  iity.  The  author  proposes  an  adaptive  model 
of  instructional  system  evaluation  in  whicn  measures  of  performance  are  taken  at  the 
completion  of  instruction  and  in  the  job  environment.  The  first  measure  is  an  indicant  of 
the  internal  state  of  the  system,  while  the  latter  is  an  indicant  of  the  impact  of  the 
system  on  consumer  needs.  Measures  of  the  operating  system  can  provide  good  measures 
of  maintenance  training  performance:  system  downtimes,  time  to  repair,  supervisor 
ratings,  and  promotion  comparisions  of  graduates  and  nongraduate  of  experimental 
programs.  In  computing  cost  savings  which  may  be  achieved  by  adoption  of  an  innovative 
training  approach,  one  should  consider  the  overall  impact  on  operation  system  perfor¬ 
mance  which  is  realized  by  having  better  trained  maintenance  personnel. 

Caro,  P.  W.  Some  current  problems  in  simulator  design,  testing  and  use  (HumRRO-PP-2- 
77).  Alexandria:  Human  Resources  Research  Organization,  March  1977. 

This  paper  discusses  some  of  the  general  and  managerial  problems  that  were  observed 
in  simulator  projects  in  the  Army,  Navy,  Air  Force,  and  Coast  Guard  and  that  decrease 
the  efficiency  of  simulator  training.  The  problems  identified  are  discussed  within  the 
appropriate  life  cycle  phase  of  a  simulator:  the  design  phase,  the  testing  phase,  or  the  use 
phase. 

Two  problems  are  associated  with  the  design  phase.  (1)  Too  often  the  simulator  user 
is  not  involved  in  the  design  process.  (2)  Although  their  purpose  is  training,  simulators  are 
designed  to  reflect  physical,  rather  than  behavioral  or  training,  models. 

Two  problems  are  associated  with  the  simulator  testing  phase.  (1)  The  testing 
conducted  to  determine  the  acceptability  of  a  simulator  is  based  on  hardware  considera- 
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tions  and  not  on  training  usefulness.  (2)  The  designer  is  given  little  feedback  on  the 
adequacy  of  the  simulator  for  training. 

Four  problems  are  discussed  in  the  simulator  use  phase.  (1)  Little  attention  has  been 
given  to  the  development  of  the  technology  of  simulator  training.  (2)  Simulator 
instructors  are  not  trained  adequately.  (3)  Simulator  training  is  typically  evaluated  in 
terms  of  rate  of  device  use,  rather  than  in  terms  of  training  benefits  or  trainee 
performance.  (4)  No  one  knows  whether  simulator  training  programs  are  cost  effective. 

Harman,  H.  H.  Designing  and  implementing  the  system  model.  In  3.  E.  Uhlaner  (Ed.), 
Psychological  Research  in  National  Defense  Today  (Tech.  Rep.  S-l).  Washington,  DC: 
U.S.  Army  Behavioral  Science  Research  Laboratory,  3une  1967. 

This  paper  covers  the  nature  and  advantages  of  simulation  in  training  exercises,  and 
presents  several  of  the  major  training  programs  employing  simulation  techniques.  It 
distinguishes  between  operations  simulation  and  symbolic  simulation.  It  lists  the 
advantages  of  simulation  techniques,  including  their  flexibility  and  economy.  The  system 
training  programs  (STP)  are  used  by  the  Air  Force  and  Army  to  illustrate  the  process  and 
usefulness  of  simulation  in  defense  training  systems. 

3ohnson,  S.  L.  Training  devices:  Physical  versus  psychological  simulation.  Proceedings  of 
the  Human  Factors  Society  19th  Annual  Meeting.  Dallas,  October  1975,  24. 

Three  major  categories  of  variables  are  involved  in  the  behavioral  aspects  of 
simulation  technology:  (1)  simulation  fidelity,  (2)  external  influences,  and  (3)  measures  of 
training  device  effectiveness.  The  concepts  of  simulation  and  training  are  discussed,  and 
the  distinction  made  between  physical  and  psychological  simulation.  The  implications  of 
psychological  simulation  on  the  variables  of  simulation  technology,  as  well  as  the  job  of 
determining  simulation  requirements  for  training  devices,  are  discussed. 

Karentz,  V.  Simulated  maintenance  training.  The  Second  Symposium  on  Cost  Effective 
Learning  Through  the  Application  of  Computer  Technology,  Washington,  DC,  February 
1973,  158-178. 

The  author  describes  how  parameters  are  defined,  transformed,  and  weighted  for 
maintenance  training  simulators.  Also,  addressed  is  how  the  resultant  training  tasks 
configure  a  training  computer  exerciser. 

Kelley,  C.  R.,  &  Wargo,  M.  3.  Adaptive  techniques  for  synthetic  flight  training  systems 
(Tech.  Rep.  NAVTRADEVCEN  68-C-0 136-1).  Orlando:  Naval  Training  Device  Center, 
October  1968. 

This  report  is  a  discussion  of  the  application  of  adaptive  training  techniques  to 
synthetic  flight  training  systems  (SFTS),  especially  to  the  2B24  training  systems.  It 
covers  three  basic  requirements  for  any  adaptive  training  system --performance  measure¬ 
ment,  adaptive  variables,  and  the  relation  of  adaptive  variables  to  measured  perform¬ 
ance- -and  the  major  problem  areas  encountered  in  the  design  of  adaptive  training 
systems.  An  example  is  given  to  illustrate  how  adaptive  training  might  be  applied  to  the 
2B24  SFTS,  and  a  plan  is  presented  for  empirically  establishing  parameter  settings, 
coefficients,  and  values  for  adaptive  features  of  the  2B24  synthetic  flight  training  system. 
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Klein,  G.  C.,  Kane,  J.  J.,  Chinn,  A.  A.,  &  Jukes,  A.  O.  Analyzing  training  device 
effectiveness  in  cases  where  test  data  is  inconclusive.  Proceedings  of  the  Human 
Factors  Society  22nd  Annual  Meeting,  Detroit,  October  1978,  152-156. 

The  paper  discusses  BDM  Corporation's  experience  in  applying  theTRAINVICE  model 
of  training  device  effectiveness  prediction  in  conjunction  with  four  CTEAs  performed  for 
the  Army.  Focus  is  upon  the  most  recent  analysis  of  the  trainer  for  the  ground/vehicular 
laser  locator  designator.  In  this  particular  analysis,  the  authors  were  forced  to  use  the 
TRAINVICE  model,  as  trainer  test  data  were  found  to  be  unreliable.  Two  versions  of  the 
TRAINVICE  model  were  applied  (Wheaton,  et  al.  1976,  &  Narva,  1978)  with  approximately 
equal  results.  The  value  in  the  TRAINVICE  analysis,  according  to  the  authors,  was  in 
providing  additional  support  for  conclusions  based  on  the  available  test  data,  thereby 
strengthening  these  conclusions.  Although  the  TRAINVICE  model  is  seen  to  offer  utility, 
such  approaches  must  be  considered  initial  efforts  in  the  development  of  methods  to 
predict  training  device  effectiveness. 

Semple,  C.  A.,  Caro,  P.  W.,  &  Seay,  D.  E.  Simulator  training  requirements  and 
effectiveness  study  (STRES).  Proceedings  of  the  Human  Factors  Society  22nd  Annual 
Meeting,  Detroit,  October  1978,  251-255. 

A  long-term  research  study  was  conducted  by  the  Air  Force,  with  participation  by  the 
Army  and  Navy  to  obtain  and  organize  data  related  to  aircrew  training  devices  (ATDs)  so 
that  ATDs  can  be  specified,  designed,  and  used  in  the  most  effective  and  efficient  manner 
within  aircrew  training  programs.  Basic  goals  of  the  study  include:  (1)  development  of 
criteria  to  match  training  requirements  to  simulator  features,  (2)  definition  of  principles 
for  the  effective  and  efficient  use  of  ATDs,  (3)  development  of  principles  for  matching 
instructional  features  to  specific  training  requirements,  and  (4)  collection  of  data  related 
to  cost  and  worth-of-ownership  of  ATDs. 

Shannon,  R.  H.  Development  of  a  training  model  for  naval  flight  students.  Proceedings  of 
the  Human  Factors  Society  18th  Annual  Meeting,  Huntsville,  AL,  October  1974,  57-60. 

This  study  was  directed  toward  the  development  of  a  part-whole  training  model  for 
naval  flight  students  separating  into  component  parts  the  maneuvers  in  presolo  training. 
This  involved:  (1)  developing  highly  detailed  descriptions  of  all  maneuvers  in  presolo 
training,  (2)  isolating  those  critical  errors  within  each  of  the  maneuvers,  (3)  performing 
two  commonality  analyses  on  the  maneuvers  (one  based  on  similar  flight  tasks  and  the 
other  on  similar  flight  errors),  and  (4)  developing  inflight/simulation  scenarios  that  could 
be  used  to  impact  upon  both  flight  tasks  and  pilot  errors.  The  results  of  this  investigation 
seem  to  indicate  the  feasibility  of  the  part-whole  model  outlined  as  being  applicable  not 
only  to  flight  training,  but  also  to  student  evaluation. 

Smode,  A.  F.,  &  Hall,  E.  R.  Translating  information  requirements  into  training  device 
fidelity  requirements.  Proceedings  of  the  Human  Factors  Society  19th  Annual  Meeting, 
Dallas,  October  1975,  33. 

When  training  needs  are  not  met  by  simulator  capability,  it  is  easy  to  assign 
responsibility  to  inadequacies  in  engineering,  design,  knowledge,  and  technique.  However, 
inaccurate  or  inadequate  information  requirements  may  be  equally  responsible  for  device 
failure.  The  quality  of  device  design  is  related  directly  to  the  amount  and  kind  of 
information  available  for  use  in  the  design  process.  A  solid  base  of  human  factors/train¬ 
ing  information  is  crucial.  The  human  factors  effort  leads  to  statements  of  the  specific 
training  needs  the  device  must  be  capable  of  meeting  and  the  conditions  for  training  that 
it  must  be  able  to  create  so  they  can  be  translated  into  design  features.  Fidelity  of 


simulation  is  constrained  by  economic,  psychological,  and  technical  factors.  The  four 
basic  levels  of  fidelity  useful  in  satisfying  defined  training  requirements  are  discussed. 

Winship,  F.  E.  Technical  publications  and  training  and  their  impact  on  cost.  Proceedings 
of  the  6th  NTEC/Industry  Conference,  Orlando,  1973. 

The  author,  while  not  discounting  the  gains  in  training  effectiveness  brought  about  by 
training  hardware,  feels  that  significant  cost  savings  may  be  realized  through  improve¬ 
ments  in  the  technical  publications  construction  process.  The  author  feels  that  these 
savings  will  take  place  if  random  access  pocket-size  publications  are  made  available,  thus 
cutting  down  on  unnecessary  replacement  of  components  and  an  even  large  savings  in 
tracking  down  information. 

5.  Job  Performance  Aids. 

Ayoub,  M.  A.,  Cole,  J.  L.,  Sakala,  M.  K.,  <5c  Smillie,  R.  J.  Final  Report:  Job  performance 
aids:  An  assessment  of  needs;  a  literature  review.  Washington,  DC:  Naval  Air  Systems 
Command,  October  1974. 

In  regard  to  the  maintenance  and  operation  of  attack  systems,  this  report  surveyed 
the  little  research  conducted  to  date  in  the  area  of  job  performance  aids.  Recommenda¬ 
tions  were  made  for  answering  the  following  questions: 

1.  What  kinds  of  information  does  an  operator  need  in  the  performance  of  a  task? 

2.  How  can  the  operator  generate  a  display  which  contains  the  required  informa¬ 
tion? 

3.  How  should  the  display  be  structured  so  that  the  operator  can  extract  the 
required  information? 

4.  How  should  an  assignment  be  organized  into  tasks? 

5.  What  basic  operations  does  the  operator  need  to  know  in  order  to  generate  and  to 
use  the  generated  information? 

6.  What  are  the  costs  and  gains  of  varying  various  parameters  in  a  job  performance 
aid  system? 

Blanchard,  R.  E.,  &  Laabs,  G.  J.  A  personnel  system  concept  with  job  performance  aiding 
as  a  principal  component:  Preliminary  model  (NPRDC  Tech.  Note  78-6).  San  Diego: 
Navy  Personnel  Research  and  Development  Center,  February  1978. 

At  present,  the  Navy  is  faced  with  rapidly  increasing  personnel  and  training  costs 
accompanied  by  declining  entry  level  skills.  Huge  amounts  of  money  invested  in  early 
training  are  lost  through  attrition.  One  possible  solution  is  to  make  training  investments 
incrementally  dependent  upon  the  individuals  shipboard  performance,  motivation,  and 
interest.  Job  performance  aiding  (JPA)  technology  offers  an  approach  that  can  make 
recruits  effective  contributors  to  shipboard  operation  and  maintenance  without  substan¬ 
tial  intial  training.  A  job  performance  aid  is  a  device  or  guide  used  on  the  job  to  provide 
the  information  necessary  for  satisfactory  job  performance.  Since  it  frees  the  user  from 
the  necessity  of  actually  learning  the  information,  training  time  can  be  reduced.  The 
purpose  of  this  study  was  to  conceptualize  a  functional  personnel  system  using  perform¬ 
ance  aiding.  Primary  objectives  of  model  development  were  (1)  to  get  individuals  on  the 
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job  as  quickly  as  possible  with  a  minimum  of  initial  training,  (2)  to  provide  for  the 
advancement  of  qualified,  career-oriented  personnel,  (3)  to  improve  the  utilization  of 
lesser-aptitude  personnel,  (4)  to  encourage  the  development  of  a  trained  career  force. 


A  general  approach  to  personnel  systems  development  was  used.  The  model 
developed  emphasized  individuals  with  4-year  enlistment  obligations.  The  model  is  in 
graphic  form  and  includes  functional  steps  and  events,  event  length,  and  estimated  time  in 
service.  The  advantages  and  disadvantages  of  the  model  are  discussed. 

Booher,  H.  R.  Job  performance  aiding  systems  technology  selection  algorithm.  Proceed¬ 
ings  of  the  Human  Factors  Society  22nd  Annual  Meeting,  Detroit,  October  1978,  158- 
162. 

The  article  presents  a  nine-step  decison  aid  for  the  selection  of  job  performance  aids 
(JPAs).  This  aid  is  seen  to  be  appropriately  used  in  early  stages  of  training  system 
development  when  trade-offs  can  be  determined  between  JPAs,  media,  and  training 
techniques.  The  decision  aid  was  based  on  available  data  relating  the  kind  of  JPA  to 
personnel  characteristics,  experience  level  of  personnel,  job  equipment  complexity,  and 
task  complexity.  The  type  of  JPA  selected  with  this  method  is  then  related  to  an 
appropriate  training  technique.  Experience  in  applying  this  decision  aid  indicates  it  is  a 
workable  approach.  Work  in  progress  will  expand  the  method  to  include  cost  trade-offs 
among  aiding,  training,  selection,  and  job  design  considerations. 

Training  Cost  Estimation 

Braby,  R.,  Micheli,  G.  S.,  Morris,  C.  L.,  <5c  Okraski,  H.  C.  Staff  study  on  cost  and  training 
effectiveness  of  proposed  training  systems  (TAEG  Rep.  No.  1).  Orlando:  T raining 
Analysis  and  Evaluation  Group,  June  1972. 

One  method  of  optimizing  training  media  allocation  on  the  basis  of  fixed  training 
effectiveness  and  a  minimum  cost  was  developed  at  Naval  Training  Equipment  Center 
(NTEC).  This  method  employs  a  digital  computer  and  linear  programming  techniques  to 
provide  the  decision  maker  with  the  information  needed  to  assess  the  costs  and  benefits  of 
alternative  training  considerations.  Specifically,  based  on  the  parameters  of  the  training 
task  and  various  training  alternatives,  the  model  identifies:  (1)  the  least  costly  of  several 
equally  effective  alternatives  of  achieving  the  training  goals,  (2)  the  alternative  expected 
to  produce  the  greatest  training  benefits  or  effectiveness  for  a  given  cost  level,  and  (3) 
the  relative  costs  of  all  alternatives  and  the  advantages  provided  by  each. 

This  model  proceeds  through  a  series  of  steps  in  which  the  necessary  parameters  are 
identified  and  systematically  incorporated  into  various  submodels.  These,  in  turn,  are 
used  by  the  computer  to  generate  the  information  needed  by  the  decision  maker  in 
selecting  the  most  appropriate  media.  Specifically,  the  task  analysis  and  incoming  skills 
and  knowledge  are  combined  to  determine  training  tasks  and  stages  of  training.  Media 
options  and  possible  substitutions  among  them  are  then  determined.  Life-cycle  cost 
factors  are  identified  for  each  alternative  media  and  analyzed  with  the  previously 
determined  data  using  linear  programming  techniques.  Ultimately,  this  provides  the 
decision  maker  with  the  information  needed  to  select  the  most  appropriate  media. 


Department  of  the  Army.  Cost  and  operational  effectiveness  analysis  in  the  materiel 
acquisition  process.  (TRADOC  Reg.  No.  11-8).  Fort  Monroe,  VA:  Headquarters, 
United  Army  Training  and  Doctrine  Command,  March  1977. 

This  regulation  outlines  the  purpose,  scope,  objectives,  and  essential  elements 
involved  in  an  assessment  of  the  cost-effectiveness  trade-offs  of  alternative  systems 
proposed  to  reduce  an  Army  force  or  mission  deficiency  for  a  given  threat  projection. 

DiGialleonardo,  F.  R.  An  approach  for  measuring  benefit  and  cost  in  management  and 
information  systems  (NPRDC  Tech.  Rep.  75-21).  San  Diego:  Navy  Personnel  Research 
and  Development  Center,  October  1974.  (AD-A014  209) 

The  author  devised  methods  for  subjectively  measuring  primarily  the  benefits  and,  to 
a  lesser  extent,  the  costs  of  elements  and  functions  of  nodes  within  a  management  and 
information  system.  This  particular  study  is  intended  for  inclusion  in  a  larger  analysis  of 
the  Navy's  manpower  planning  system. 

Doughty,  P.  L.,  Stern,  H.  W.,  <5c  Thompson,  C.  Guidelines  for  cost  effectiveness  analysis 
for  Navy  training  and  education  (NPRDC  Spec.  Rep.  76TQ-12).  San  Diego:  Navy 
Personnel  Research  and  Development  Center,  3uly  1976. 

In  this  report  a  very  thorough  approach  was  presented  which  incorporated  systems 
and  cost  analysis  techniques  into  the  analysis  of  the  cost-effectiveness  for  Navy  training 
and  education.  The  authors  proposed  that  alternatives  should  be  considered  not  only  on 
the  basis  of  training  products,  but  also  on  the  basis  of  resources  required  to  support  each 
alternative.  Additionally,  as  the  training  program  is  being  developed  and  administered, 
constant  consideration  should  be  given  to  revising  the  assumptions  used  to  select  the 
particular  alternative.  As  these  assumptions  change,  or  are  invalidated,  alternative 
training  system  strategies  should  be  considered. 

Three  uses  of  cost-effectiveness  analysis  are  identified:  (1)  descriptive  analysis, 
which  is  intended  to  assess  ongoing  or  completed  programs;  (2)  comparative  analysis, 
which  involves  the  comparison  of  various  alternative  approaches;  (3)  predictive  analysis, 
which  involves  the  determination  of  costs  and  benefits  of  the  proposed  alternatives. 
Comparative  analysis  may  involve  both  descriptive  analyses  (after-the-fact  comparisons) 
and  predictive  analyses  (a  priori  comparison  of  alternatives). 

Two  basic  alternative  approaches  to  cost-effectiveness  analysis  were  presented: 
fixed  effectiveness  and  fixed  cost.  The  fixed-effectiveness  approach  implies  that  the 
level  of  training  effectiveness  is  fixed  (i.e.,  better  training  is  unnecessary  and  less 
training  is  intolerable).  In  this  case,  the  objective  is  to  minimize  overall  cost.  In  the 
fixed-cost  approach,  the  budget-level  is  fixed  and  the  objective  is  to  maximize  the 
training  system's  effectiveness.  In  many  cases,  the  actual  approach  taken  permits 
cost/benefits  trade-offs  within  limits;  however,  the  concepts  provide  a  valuable  frame¬ 
work. 

The  six  proposed  cost  analysis  techniques  listed  in  descending  order  of  sophistication 
are:  (1)  cost  benefit,  (2)  cost-effectiveness,  (3)  unit  cost,  (4)  cardinal  weighting,  (5) 
ordinal  ranking,  and  (6)  best-guess  decision  making.  Each  of  these  techniques  requires 
different  data  and  analyses  and,  therefore,  each  is  appropriate  in  different  situations. 

The  authors  also  provide  a  taxonomy  of  training  methods  that  describe  the  instruc¬ 
tional  process  in  sufficient  detail  to  permit  adequate  cost  analysis.  The  general 
categories  are: 


1. 

2. 

3. 

4. 

5. 


Passive  presentation  (e.g.,  verbal  delivery,  written  delivery). 
Student/instructor  interactive  presentations  (e.g.,  discussion). 
Point  and  graphic  methods  (e.g.,  programmed  instruction,  CA1). 
Functional  applications  (e.g.,  laboratories,  simulation). 

Course  management  (e.g.,  computer-managed  instruction). 


In  addition  to  these  methods,  the  following  variables  are  important  in  describing  a  training 
problem:  student  throughput,  appropriateness  of  instructional  methods,  and  student 

management. 

Finally,  the  authors  identified  five  cost  categories  and  four  cost  driving  activities. 
The  cost  categories  are:  (1)  personnel,  (2)  hardware,  (3)  software,  (4)  facilities,  and  (5) 
overhead  costs.  Activities  that  might  require  resources  from  any  of  the  cost  categories 
and,  therefore,  drive  these  costs  are:  (1)  research  and  development,  (2)  investment  and 
production,  (3)  replacement,  and  (4)  operation. 

Hazer,  K.,  Jr.,  &  Ringler,  D.  Applicability  of  design-to-cost  to  simulator  acquisition. 
W right-Pa tterson  Air  Force  Base,  OH:  Air  Force  Institute  of  Technology,  June  1976. 
(AD-A030  231) 

The  authors  assembled  a  set  of  criteria  thought  to  be  relevant  to  constructing  a 
decision  model  for  implementation  of  the  design- to-cost  (DTC)  concept  in  simulator 
acquisition.  Personnel  familiar  with  the  DTC  concept  then  rated  25  of  the  26 
characteristics  as  viable  decision  elements  for  the  applicability  of  the  DTC  concept 
toward  weapon  acquisition  programs.  Personnel  from  the  Simulator  Systems  Program 
Office  (W  right-Pa  tterson  AFB,  OH)  then  judged  the  relevance  of  the  25  criteria  as  to 
their  applicability  toward  simulator  acquisition.  Of  the  25  characteristics,  15  were  judged 
to  be  relevant,  6  marginally  relevant  depending  on  the  size  of  the  project,  and  4  were 
thought  to  be  irrelevant  in  simulator  acquisition. 

The  authors  concluded  that  DTC  concepts  not  be  implemented  in  simulator  acquisi¬ 
tion  systems  because  (1)  two  of  the  characteristics  judged  not  applicable  ranked  relatively 
high  in  the  mean  values  rating  and  (2)  a  particular  simulator  considered  had  brief 
conceptual  and  definition  phases,  thus  not  supporting  a  use  of  the  DTC  concept.  However, 
the  use  of  DTC  in  simulator  acquisition  might  be  considered  relevant  if  the  simulator 
acquisition  involves  the  development  of  a  new  simulation  system  or  related  component 
that  is  technologically  feasible,  but  not  available  in  private  industry.  Al'so,  judged  a 
viable  instance  of  DTC  application  is  when  production  techniques  are  used  t;o  produce  a 
large  number  of  simulation  systems  or  components,  and  production  costs  are  relatively 
large  in  comparison  to  other  acquisition  costs. 

Johnson,  S.  L.  Establishing  training  criteria  on  an  economic  basis,  Proceedings  of  the 
Ninth  NTEC/Industry  Conference,  Orlando:  November  1976. 

Johnson  cast  the  problem  of  developing  training  programs  on  an  economic  basis 
within  a  quality  control  paradigm.  He  viewed  training  as  a  method  of  increasing  the 
reliability  of  one  component  in  a  man-machine  system.  As  with  the  quality  control 
problem,  the  training  analyst  must  attempt  to  minimize  the  risk  of  wasting  resources  (i.e., 
over-training  the  human),  while  also  minimizing  the  risk  of  not  providing  enough  training, 
and,  thus,  graduating  unqualified  personnel  (producers'  and  consumers'  risk,  respectively). 
It  is  a  problem  of  establishing  an  acceptable  outgoing  quality  level  of  a  training  program 
on  the  basis  of  economics. 
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He  identified  the  following  parameters  for  use  in  his  model: 


Pqcc  =  probability  a  situation  will  occur 

PQ  =  probability  a  trainee  will  be  able  to  perform  the  task  without  training 

P  =  probability  a  trainee  will  be  able  to  perform  the  task  after  training 
time  t 

K  =  cost  of  t  units  of  training 

K  =  cost  of  a  failure  to  perform  the  task 

OCG 

Using  these  parameters,  the  probability  the  human  will  be  able  to  perform  the  task  after 
training  time  t  is  (P  -  P  )  (P„  J.  The  value  or  payoff  for  the  training  becomes  (P  -  P  ) 

T  O  OCC  X  O 

^Pocc^  ^Kocc^‘  Similarly,  the  cost  of  training  is  (K  At)  (t).  Then,  according  to  this  model, 
the  break-even  point  is  when  the  value  of  the  training  equals  its  cost.  That  is  when 


(Pt-  Po>  <Pocc>  ^pcc* 

(KAtMt) 


=  1 


Hypothetical  examples  are  used  in  this  paper  to  illustrate  the  applicability  of  this 
model  to  the  development  of  training.  The  point  is  well  made  that  cost-effective  training 
programs  can  be  established  using  the  same  concepts  that  successfully  ensure  an  adequate 
level  of  quality  in  an  industrial  production  environment. 


Jorgensen,  C.  C.,  ic  Hoffer,  P.  Prediction  of  training  programs  for  use  in  cost  training 
effectiveness  analysis.  El  Paso:  U.S.  Army  Research  Institute  for  the  Social  and 
Behavioral  Sciences,  Fort  Bliss  Field  Unit,  January  1978. 

Jorgensen  and  Hoffer  developed  a  technique  for  selecting  the  most  cost-effective 
media  based  on  a  comparison  of  training  objective  media  requirements  and  candidate 
media  characteristics.  They  state  that  estimation  of  media  cost  presents  relatively  minor 
problems  when  compared  to  the  estimation  of  its  training  effectiveness.  The  primary  goal 
of  the  report  is  to  develop  a  technique  for  assessing  the  effectiveness  of  media 
alternatives  for  the  training  of  specific  tasks. 

Their  basic  approach  for  determining  media  effectiveness  involves  the  development 
of  a  set  of  matrices  that  describe:  (1)  the  media  requirements  from  the  standpoint  of 
most  desirable  instructional  delivery  for  each  task  and  (2)  media  characteristics  from  the 
standpoint  of  which  characteristics  are  available  in  which  media. 

The  columns  of  both  matrices  represent  a  set  of  variables  describing  the  media  in 
terms  of  capability  and  availability  (e.g.,  color  availability  and  characteristics  of  audio 
cues).  The  rows  in  the  task  matrix  represent  the  various  tasks  to  be  trained.  In  this 
matrix,  the  values  within  each  cell  represent  the  desirability  of  the  characteristic 
(identified  by  the  column  number)  for  training  of  task  (identified  by  the  row  number).  In 
the  media  characteristics  matrix,  the  rows  represent  the  various  media  alternatives  (e.g., 
sound-slide,  workbook,  classroom,  and  simulator).  In  this  matrix,  the  value  within  each 
cell  represents  the  availability  or  unavailability  of  a  particular  characteristic  for  a 
particular  media. 


By  comparison  of  the  two  matrices  through  matrix  algebra,  an  analysis  is  made  to 
determine  which  media  has  the  most  desirable  combination  of  characteristics  for  the 


training  of  each  task.  The  selected  media  is  designated  as  the  optimal  media  from  a 
training  effectiveness  standpoint.  The  relative  desirability  of  alternate  media  can  then  be 
assessed  by  comparing  the  number  of  lacking  and  optimum  characteristics.  This 
difference;  the  authors  proposed,  should  be  a  quantified  measure  of  training  effectiveness 
which  could  be  used  as  the  denominator  in  the  cost-effectiveness  ratio.  Although  the 
technique  oversimplified  the  concept  of  media  training  effectiveness,  it  does  take  a 
significant  step  toward  the  quantification  of  the  media  selection  process. 

Martin,  J.  I.,  Koehler,  E.  A.,  <Sc  Mairs,  L.  S.  Life  cycle  Navy  enlisted  billet  costs- -FY78, 
(NPRDC  Spec.  Rep.  78-14).  San  Diego:  Navy  Personnel  Research  and  Development 
Center,  July  1978. 

The  objective  of  this  effort  was  to  provide  program  managers  and  systems  engineers 
with  current  data  on  life  cycle  costs  of  Navy  enlisted  personnel  for  1-,  5-,  10-,  15-,  and 
20-year  periods.  These  costs  are  provided  by  rating  and  pay  grade.  Availability  of  such 
data  should  enable  managers  to  predict  life  cycle  costs  of  operator  and  maintenance 
personnel.  Therefore,  candidate  system  approaches  may  be  more  easily  compared  in 
terms  of  their  manpower  requirements.  Furthermore,  comparisons  may  also  consider 
hardware  and  software  costs  associated  with  each  manpower  concept. 

Life  cycle  personnel  costs  include  direct  costs,  training  and  retirement  costs,  and 
overhead  costs  associated  with  all  personnel,  regardless  of  rating  and  pay  grade.  It  is 
presently  anticipated  that  life  cycle  Navy  enlisted  billet  costs  will  be  recomputed, 
updated,  and  published  at  yearly  intervals.  Variables  included  in  computation  are  billet 
cost,  number  of  years  in  billet  life,  discount  rate,  and  billet  cost  for  first  year. 

Okraski,  H.  C.,  &  Parrish,  W.  F.,  Jr.  Acquisition  cost  estimating  using  simulation  (TAEG 
Tech.  Memo  75-4).  Orlando:  Training  Analysis  and  Evaluation  Group,  September  1975. 
(AD-A015  624) 

The  acquisition-cost-estimating-using-simulation  (ACES)  model  developed  by  the 
Training  Analysis  and  Evaluation  Group  (TAEG),  which  is  designed  to  determine  the  most 
probable  costs  of  any  hardware  and/or  software  system,  is  especially  useful  when 
uncertainty  exists  in  the  design  approach.  In  using  ACES,  the  subsystems  of  a  given 
system  must  be  identified  and  then  the  possible  technical  approaches  for  realizing  each 
subsystem  are  determined.  Once  this  information  is  available,  probability  of  occurrence 
for  any  approach  can  be  assigned  and  cost  estimates  for  each  approach  calculated.  ACES 
works  by  randomly  selecting  subsystem  costs  according  to  the  probability  of  occureence, 
and  through  repeated  iterations  (as  many  as  5000  or  6000)  creating  patterns  of  possible 
cost  outcomes.  Upon  completion  of  simulation,  a  graph  is  plotted  of  probability  versus 
least  system  cost,  as  well  as  a  histogram  of  system  cost  distribution.  These  provide  the 
program  manager  with  a  good  feel  for  possible  costs.  The  program  is  availabe  to  the 
public. 

Skeen,  J.  R.,  <5c  Jackson,  A.  E.  A  life  cycle  training  cost  model.  San  Diego:  Navy 
Personnel  Research  and  Development  Center  (in  press). 

This  paper  develops  a  general  model  to  be  used  in  determining  the  cost  of  training 
required  throughout  the  life  cycle  of  a  Navy  weapons  system.  System  training  takes  place 
in  a  context  of  weapon  system  development  model,  which  is  described  to  indicate  both  the 
major  categories  of  personnel  who  will  require  training  and  the  timing  of  that  training. 

A  training  course  was  chosen  as  the  focus  of  the  model,  because  it  is  traditionally 
understood  by  both  training  managers  and  instructional  technologists.  The  overall  nature 
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and  life  cycle  of  a  course  are  discussed,  and  the  activities  involved  in  initial  course 
development  are  analyzed. 

The  end  result  of  this  analysis  is  a  definition  of  a  course  in  terms  of  a  three- 
dimensional  matrix  whose  axes  are:  (1)  the  four  development  phases  (requirement 
specification  phase,  planning  and  design  phase,  development  phase,  and  operation  and 
control  phase),  (2)  the  six  element  costs  (personnel,  facilities,  equipment,  materials  and 
supplies,  transportation,  and  miscellaneous),  and  (3)  the  eight  course  activities  (course 
content/media,  student,  instructor,  hardware,  facilities,  evaluation,  management,  and 
support). 

The  training  life  cycle  cost  model  thus  developed  is  used  to  (l)  estimate  the  number 
of  different  courses  required,  (2)  estimate  the  number  of  repetitions  and  probable 
modifications,  (3)  estimate  the  costs  of  activities  for  each  course  development  phase  for 
each  repetition,  (4)  sum  across  repetitions  to  obtain  total  estimated  course  cost,  and  (5) 
sum  the  costs  of  all  individual  courses  to  obtain  the  total  estimated  system  training  cost. 
The  model  is  intended  to  be  descriptive,  rather  than  directive,  and  to  provide  a  single 
coherent  point  of  reference  for  use  by  system  training  manager,  instructional  technolo¬ 
gists,  and  training  cost  accountants. 

Swope,  W.  M.  A  primer  on  economic  analysis  for  naval  training  systems  (Tech.  Rep. 

TAEG-30).  Orlando:  Training  Analysis  and  Evaluation  Group,  March  1976. 

A  method  for  performing  economic  analysis  is  presented  in  a  guidebook  format  for 
use  by  the  Naval  Education  and  Training  Command.  The  primary  function  of  this 
guidebook  is  to  describe  the  components  of  economic  analysis  most  relevant  to  training- 
system  decison  makers.  Several  points  brought  out  in  the  report  are  of  particular  interest 
to  persons  involved  in  preliminary  estimation  of  training  system  costs.  During  the 
identification  of  resource  requirements,  three  resource  classes  were  identified:  (1) 
research  and  development  costs,  (2)  implementation  costs,  and  (3)  operation  and 
maintenance  costs.  Additionally,  the  following  six  subclasses  of  resources  were 
identified:  (1)  instructional  material  development,  (2)  facilities,  (3)  equipment  (4) 

expendable  supplies,  (5)  personnel,  and  (6)  student  salaries.  Presumably,  if  all  of  these 
classes  and  subclasses  of  resource  requirements  are  adequately  considered,  an  accurate 
estimate  of  total  resource  requirements  will  be  obtained. 

This  report  identifies  several  specific  methods  for  obtaining  resource  requirement 
estimates.  The  first  method  involves  the  development  of  cost  estimating  relationships 
(CERs).  These  are  the  functional  relationships  between  physical  and  performance 
attributes  of  a  training  system.  Another  method,  termed  the  industrial  engineering 
method,  involves  the  breakdown  of  the  training  system  into  elements  or  components  (e.g., 
skills)  and  estimating  the  costs  of  each  component  (e.g.,  resource  requirements).  These 
methods,  will  greatly  assist  in  the  development  of  any  handbook  for  resource  requirement 
estimation. 

The  report  discusses  the  difficulties  associated  with  estimating  the  effectiveness  or 
benefits  of  training  system  alternatives.  This  aspect  of  training  system  analysis  clearly 
presents  significant  difficulties  with  respect  to  cost/benefit  analysis. 

Other  concepts  relevant  to  the  issue  of  long-term  cost  estimation  of  training  systems 
are  also  presented.  First,  the  adoption  of  advanced  educational  technology  must  always 
be  considered  by  system  designers.  These  advancements  may  provide  the  most  efficient 
means  of  meeting  the  training  objectives.  Second,  the  possibilities  of  reorganizing  small 
similar  training  groups  into  larger  training  groups  should  be  considered  in  order  to  take 


advantage  of  the  economies  associated  with  larger  scale  operations.  Third,  for  the 
development  of  new  weapons  systems,  the  incorporation  of  existing  large  training 
programs  should  be  considered. 


Swope,  W.  M.,  &  Cordell,  C.  C.  Training  resource  classifications:  Direct- indirect  and 
fixed-variable  cost  categories  (TAEG  Tech.  Memo  76-0.  Orlando:  Training  Analysis 
and  Evaluation  Group,  June  1976.  (AD-A029  179) 

Swope  and  Cordell  investigate  training  resource  costs  to  determine  their  relative 
sensitivity  to  lead  times.  Also,  the  authors  developed  a  scheme  for  determining  if 
particular  cost  is:  (l)  fixed  or  variable  and  (2)  indirect  or  direct. 

Swope,  W.  M.,  &  Green.  A  guidebook  for  economic  analysis  in  the  Naval  Education  and 
Training  Command  (TAEG  Rep.  55).  Orlando:  Training  Analysis  and  Evaluation  Group, 
April  1978. 

This  guidebook  for  performing  economic  analyses  of  training  systems  to  aid  decision 
makers  of  the  Naval  Education  and  Training  Command  is  somewhat  more  procedurally 
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